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TrueNAS 


UNIFIED. SCALABLE. FLEXIBLE. 








Across all industries the demands of data infrastructure have soared to new heights. 


As Capacity requirements continue to rise at an ever-increasing rate, performance must not be compromised. The hybrid 
architecture and advanced software capabilities of the TrueNAS appliance enable users to be more agile, effectively 
manage the explosion of unstructured data and deploy a centralized information storage infrastructure. Whether it’s 
backing virtual machines, business applications, or web services, there’s a TrueNAS appliance suited to the task. 


TrueNAS™ Storage Appliances: Harness The Cloud 


iXsystems’ TrueNAS Appliances offer scalable high-throughput, low latency storage 


All TrueNAS Storage Appliances feature the Intel® Xeon” Processors 5600 series, powering the fastest data transfer 
speeds and lowest latency possible. TrueNAS appliances come in three lines: Performance, Archiver, & High Availability. 
High-performance, high-capacity ioMemory modules from Fusion-io are available in the TrueNAS Enterprise, Ultimate, 


and Archiver Pro models. 
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Key Features: 








« One or Two Six-Core Intel® Xeon® Processors 
5600 series 


« Share Data over CIFS, NFS and iSCSI 

« Hybrid storage pool increases performance and 
decreases energy footprint 

¢ 128-bit ZFS file system with up to triple parity 
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Call iXsystems toll free or visit our website today! 
1-855-GREP-4-IX | www.iXsystems.com 
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Dear Readers, : 
he March issue starts with Rob’s column. Bei, MEIC 
) to send us your opinions about it. Than, in What’s New 
section you will find Juraj Sipos article on his newrelease of 
MaheshaBSD Server. In his article he ine ah xe OIReNS 
PETOOO CORONA Mian Ore. ei al ea 
Off a USB memory stick without any installasi6Nn and configuration 

procedures. 

Next, is Luca’s article on reacting to kernel panic. A very 


detailed, long article containing many codes. If you come mee 
any difficulties when following his steps, just write to him and; ee ne 


your questions. He will be happy to assist you. 
The third part of Rob’s series on FreeBSD Programming Primer 


Xe L-IETEClIOR LCM HILA Me LIK NIT VA colat ime alice) (eg Ma Mae ey ee 


we will look at the code structure, program flow and we will show 
how to embed CSS and Javascript in our pages. We will also start 


using SQL to dynamically generate web pages. SMM ah | 


you can learn much ouvet it. eee 


In section dedicated to security, you will find ie 


part of Hardening FreeBSD with TrustedBSD and Mandatory 
Access Controls. It covers the basic configuration of the mac_ifoff, 
mac_portacl, and MAC LOMAC modules. We hope you enjoyed 
MORESO EES 


Oye) . 
We are preparing for you something really special for April issue — 
the whole issue will be dedicated to FreeNAS! Don't miss it! 


Wish you enjoy the read! 
Patrycja Przybylowicz 
Editor of BSD Magazine 
& BSD Team 
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Let’s Talk 


OGSchrodingers Cat, Tri-states, and the 
Menace of Metrics 
By Rob Somerville 
There is continual pressure in our metric driven industry 
to deliver. The scope triangle defines time, cost and qual- 
ity as the boundaries that limit any given project. Yet even 
with this wisdom, projects fail customer expectations, run 
late or exceed budget. Why? 


What’s New 


OSMaheshaBsSD Server Second Edition 

Has Been Released... 

By Juraj Sipos 
The first article about MaheshaBSD Server was published 
in the 2012 August issue of the BSD Magazine. It is a turn- 
key FTP/WWW Server now also with WordPress running 
immediately off a USB memory stick without any instal- 
lation and configuration procedures. It has many advan- 
tages, too. Presently, there is not such a thing available 
in the Open Source world - that is, a preconfigured Word- 
Press Server running off a writable USB memory stick. 
Particularly WordPress is the news in this release. To set 
up MySQL and WordPress in FreeBSD is a difficult task 
for the newbies. Thus, this software has also an educa- 
tional purpose. 


How To 


12 Reacting to Panic 
By Luca Ferrari 
FreeBSD provides a very powerful mechanism for report- 
ing dramatic error conditions that are known as kernel 
panic. A kernel panic is when the system does not know 
how to proceed any further, and instead of making the 
disks or the memory or any other component incosistent, 
it raises a panic that is an explicit request for help. Fortu- 
nately, FreeBSD is a secure and stable operating sys- 
tem, and this means that panics are not frequent and, 
on the other hand, the system is able to start over 
automatically to provide service continuity. However, 
knowing what a panic is and how to handle informa- 
tion about it, as well as Knowing when, in kernel or 
device driver development, it does make sense to 
issue a panic, which is a valuable knowledge for a 

system administrator. 
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Admin 


<3 OF reeBSD Programming Primer Part 3 

By Rob Somerville 
In the third part of our series on programming, we will look 
at code structure, program flow and how to embed CSS 
and Javascript in our pages. We will also start using SQL 
to dynamically generate web pages. 


security 


41=Hardening FreeBSD with TrustedBSD 

and Mandatory Access Controls (MAC) 

Part 5 

By Michael Shirk 
Most system administrators understand the need to lock 
down permissions for files and applications. In addition to 
these configuration options on FreeBSD, there are fea- 
tures provided by TrustedBSD that add additional lay- 
ers of specific security controls to fine tune the operating 
system for multilevel security. For the purpose of this ar- 
ticle, support will be loaded in with kernel modules already 
available with FreeBSD 9.1. Part 5 of the TrustedBSD se- 
ries will cover the basic configuration of the mac_ifoff, 
mac_portacl, and MAC LOMAC modules. 
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Schrodingers Cat, 
Tri-states, and the 
Menace of Metrics 


There is continual pressure in our metric driven industry to deliver. 
The scope triangle defines time, cost and quality as the boundaries 
that limit any given project. Yet even with this wisdom, projects fail 
customer expectations, run late or exceed budget. Why? 





tems, the law of Schrodingers cat comes into effect and by the sheer 
fact of measuring the system, the corresponding result becomes in- 
accurate. | may be stretching a point here, but bear with me. There are 
metrics, and there are metrics. | can take a ruler and measure the width 
of my keyboard (75 mm), and taking a reasoned view | would not expect 
the plastic of the case to vary in length unless the it was exposed to ex- 
treme heat — in which case it would either expand or contract depending 
on the plastic and thermodynamic properties of the material measured. 
| have less confidence in the metal tape measure, my ageing eyesight 
and the shape of they keyboard as it is slightly bevelled at the end and 
the tape measure is flexible. If | did not measure the width at exactly 90 
degrees, the true measurement could be for argument sake, 74-76 mm. 
So here lies the first law of measurement, in that we have to accept a de- 
gree of tolerance or error thereof. | 
My keyboard is an inanimate object, apart from the life | bestow upon 
it by supplying it power via USB, pressing the keys and giving it the oc- 
casional clean, it has no choice but to be measured. Both my sanity and 
reality itself would be challenged if the keyboard when presented with 
the measuring tape decided to move, change shape, or otherwise reb- 
el against measurement. Yet this fact seems to escape some managers ¢ 
when dealing with complex systems. The levels of tolerance and error 
start becoming so great that the measuring itself becomes the reality, 
and the facts on the ground statistical aberration. 
Take Time Attendance and Management Systems (TAMS). TAMS in 
itself, no doubt, was designed with noble intentions in mind. By identify- 
ing absentee personnel, those that were deliberately trying to defraud 
the system would be quickly identified, yet as always for the ingenious 
there is a way to load the dice. Get a colleague to clock in for you, and 
provided you are both not caught, then cheating the system becomes 
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| believe resolutely that as soon as you start measuring complex sys- 





MAGAZINE 











Schrodingers Cat, Tri-states, and the Menace of Metrics 


quite attractive. However, TAMS has evolved, and using 
the Bradford Factor, the number of absentees is given a 
higher negative loading than an sustained period of ab- 
sence, so you can end up in the scenario where statisti- 
cally it is more beneficial to have 5 consecutive sick days 
than 2 or 3 days over a long period. This enters the arena 
of the law of unintended consequences, as the statistically 
astute will quickly realise that “throwing a sickie” is more 
acceptable when the period is extended. In other words, a 
measurement that was designed to improve efficiency in 
reality empowers inefficiency. Humans are dynamic and 
do not like being measured. Schrddingers cat is alive and 
well in a human resources department near you today. 

That illustration aside, take the amount and time and 
effort it requires to manage and control the quality of the 
data in TAMS. | have encountered more time manage- 
ment systems in my career than | would wish on my worst 
enemy, and pretty much all of them are appalling. Inevita- 
bly, | end up documenting every nuance of my day that | 
have to spend time on a Friday afternoon inputting every 
hour | worked, what | did during the week and why | did tt. 
What is so nihilistic about this whole process Is that it is so 
counter productive — the time spent nurturing TAMS would 
be so much better spent dealing with that customer, fixing 
that bug or maybe just staring out the window. And no, 
| don’t get paid overtime. TAMS cannot measure quality 
of attendance, and by that sheer fact alone it defeats the 
object of the exercise. And lets not even raise the ethical 
debate that an employer should trust his employees in the 
first place (unless of course as a business model they re- 
quire a level of the dark side in which case the employees 
must be strictly managed). 

The problem with measurement is that like the letter ver- 
sus the spirit of the law, we can quickly become prison- 
ers of the system. In any given project, there will always 
be the temptation to counter project creep by getting ad- 
ditional contractors on board. That in itself is not a bad 
solution, provided that the contractors can hit the ground 
running, intimately Know what is expected and can pro- 
vide the necessary skill set without stealing time from key 
personnel on the project. Sadly, so often the team synergy 
is just not there. In my years of working in IT | have rarely 
rejoiced at the thought of liaising with 3rd party suppliers, 
and | have experienced the gamut from the IBM booted 
and suited to the ripped jeans-wearing anarchist on my 
travels. The sad fact is that the latter tends to perform 
better than the former (mainly because they hate project 
managers and a strong bond develops from the mutual 
distrust). There is this terrible period of hand holding, try- 
ing to balance internal politics versus external commercial 
reality and in the end this feeling of gloom that as project 
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tech | am going to be left holding the baby. Yet the triangle 
of time, quality and cost sets the agenda, so | resign my- 
self against every bone in my body that quality will go out 
the window as time is immutable and that deadline looms. 
Let the bean counters take the responsibility for the cost, 
but the project that could have been so much better enters 
the march of death and | get disappointed that the result 
could have been excellent rather than just average. Most 
engineers would get cynical and depressed at this point, 
but it is important to keep the passion and creativity alive. 

This brings us nicely to the philosophy of tri-state. When 
confronted with the harsh reality of distorted statistics, to 
maintain our integrity we must adopt a zen like approach 
and confound those that seek to measure every jot and 
tittle and kill productivity and creativity in the process. By 
all means fill in the time sheet, document that code, check 
that server you know will never fall over and stay within 
the rules of the system. However, when asked what your 
secret is return NULL. Even the statisticians have to ad- 
mit that NULL means no statistical significance exists in 
a set of given observations. Schrodingers cat is neither 
here nor there, and there is an ethereal quality about the 
beast. As engineers who create out of nothing, we must 
never forget that we bring magic to the black and white of 
numbers. 


ROB SOMERVILLE 

Rob Somerville has been passionate about technology since his 
early teens. A keen advocate of open systems since the mid eight- 
ies, he has worked in many corporate sectors including finance, 
automotive, airlines, government and media in a variety of roles 
from technical support, system administrator, developer, systems 
integrator and IT manager. He has moved on from CP/M and nixie 
tubes but keeps a soldering iron handy just in case. 
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WHAT’S NEW 


MaheshaBSD 


Server Second Edition Has Been Released... 





The Moon is the symbol of Manasa, the Hindu Tantric Serpent 
Goddess. | chose this Goddess because | believe that not only 
Unix, but also Tantrism is an aspect of networking. The Full Moon 
was here on February 25, 2013 - the date when the version 2 of 
MaheshaBSD Server was released. 


What you will learn... What you should know... 


¢« How toruna simple and smart FTP/WWW and WordPress server « Some knowledge on networking and... Tantrism 


lished in the 2012 August issue of the BSD Mag 
(http://osdmag.org/magazine/1 809-tuning-zfs-on- 
freebsd). The story behind MaheshaBSD Server is very 
well covered in the above-mentioned article. But for those 
who are impatient —- MaheshaBSD Server is primarily an 
Intranet server. It is a turnkey FTP/WWW Server now also 
with WordPress running immediately off a USB memory 
stick without any installation/configuration procedures. It 
has many advantages, too. Presently, there is not such a 
thing available in the Open Source world — that is, a pre- 
configured WordPress Server running off a writable USB 
memory stick. Particularly WordPress is the news in this 
release. To set up MySQL and WordPress (WordPress 
needs MySQL) in FreeBSD is a difficult task for the new- 
bies. Thus, this software has also an educational purpose. 
MaheshaBSD Server is built on MaheshaBSD. This means 
that it has all its functionality (Linux emulation, anonymity, 
VNC Server, X Window, Text-To-Speech software, etc.). 


7 he first article about MaheshaBSD Server was pub- 


System Requirements 


¢ Memory (RAM) used by MaheshaBSD Server: 





mfsroot (root directory, /dev/md0) - size 54 MB; 6,2 MB free : 
/tmp - size 140 MB Figure 1. iene the Goddess of aires In Hindu mythology, cobra 
swap ~ size 100 vB means power and that is what FreeBSD means 
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¢ The minimum memory requirements are 360 MB 
RAM (look into the “swapme?’ scripts in /root/bin and 
set up your own swap size). 

¢ A 4 GB USB flash drive minimum (or a USB hard 
drive of a size larger than 4 GB). 


There are not many free online FTP servers today. Most 
of them (if not all) have restrictions and bypassing them 
requires payment. With MaheshaBSD Server, you may 
use any hard disk (NTFS/FAT32 is not a problem ei- 
ther) and set up your own FTP/WordPress server. A be- 
ginning cameraman may put a link on his personal web- 
site to his home FTP/WordPress server with tons of Gi- 
gabytes (videos, MP3’s, etc.) of data and share the files 
with anybody in the world. 

Portability is just another advantageous feature — with 
a straightforward FTP/WWW server (WordPress runs on 
WWW) you will start your work anywhere in the world in 
a few seconds with the same files you have had in Brazil 
after you moved to Canada. 

lf you configure Samba (it is installed in this software), 
MaheshaBSD Server can also be used for many things 
Windows users are accustomed to — for example, you will 
have a cheap data storage solution accessible via Sam- 
ba. Many companies need data storage for their industrial 


by AE hitp:imanssa/wordpress/ ~| 


-oryerd stop 


Bookmarks ESanciens ETemp E3comp E9map ESxeath Jc: Fsearch Sw pic Fs! 


Hello world! 2 


Posted on January 21, 2013 


Welcome to MaheshaBSD Server. Log in (username: admin, password: 
manasai1) to WordPress and start your turnkey Intranetinternet blog right 
now — go to (Posts) Al Posts and choose Hello World to edit this page. Add 
pictures and let all see them later in your posts. 


This thing runs on FreeBSD 9.0-RELEASE. The main goal of MaheshaBsD 
Server is to provide users with an easy to use turnkey Intranet and 
home server (and turnkey education). You can, too, open it to the world. 
Edit your hosts file in Unix (/etC/Nosts), its Content does not differ from Windows 
(you will find it in C’winntsystem32\drivers\etc) — just add the following line in it 
and replace 192.168.1.200 with your actual local/public IP (to get your public 
URL, see whatismyip.com): 192.168.1.200 manasa - FTP server will be then 
available at the following link ftp://manasa, where ‘you will find, too, the 
documentation for this thing. To check this web server's functionality, click on 
this link. Manage users from inside your Admin account (Users > Add New). 
When you allow registration of users in Settings via e-mail, password e-mailing 
will not work. It will be better to manage users under your strict administrator's 
eyes. Donate to this project! 


And subscribe to New Dawn, the world’s (and Australia’s) most unusual 
magazine! 


Figure 2. Log in, edit Hello World and put your company logo 
wherever you like 
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cameras, etc. MaheshaBSD Server is a server, so users 
do not need any special knowledge of Unix. They will work 
with it the same way they work daily with the Internet. 
MaheshaBSD Server has a remote admin — Webmin. 
Change passwords and administer this software remotely. 
Expandability — if you are not satisfied with a particular 
package (for security reasons, bugs, etc.). In this software, 
you simply uninstall or upgrade it. As this thing is writable, 
customize it to your specific needs (install packages of your 
choice, make your own mail server, etc.). For example, to 
upgrade WordPress, all you need is to copy a new version 
of WordPress into /usr/local/www/wordpress. To set up your 
own mail server, use iRedMail for FreeBSD (htto:/Avww. irea- 
mail.org/install_iredmail_on_freebsd.html). Tne USB image 
has 1.4 GB of free space. | can supply larger images upon 
request. The software is VMware friendly, thus you may run 
MaheshaBSD Server in Windows as any other application. 


Quick Start 

Boot. To log in to your WordPress account, put the fol- 
lowing entry into your hosts file (/etc/nosts in Unix; C:\ 
winnt\system32\drivers\etc in Windows) in Unix or Win- 
dows (W2K, XP, Windows Vista, 7 & 8): 


your IP manasa 


like this: 

192.168.1.200 manasa 
OR 

your ublic ip manasa 


Eiireot's Xdesktop (ocalerett) 
- r 
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Figure 3. phpMyAdmin runs both locally and remotely via TightVNC 
as all MaheshaBSD Server's GUI applications 
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Then go to the URL http:/”manasa/wordpress and follow 
the instructions on the introductory page. Apache serv- 
er runs on: 


¢ http://manasa 

¢ FTP server runs on: 

¢ ftp://manasa (where you will also find the full docu- 
mentation for this thing) 


The password for root is “freebsdserver”. To operate the 
MaheshaBSD Server's FTP server immediately, log in 
as user vsftpd (passwords are in /home/guest5/passes. 
txt, SO you must first log in as guest5 with password 
guest6 to fetch all these passwords). Then, copy any- 
thing into /home/vsftpd/ftp directory (via SFTP — Win- 
dows users should use free programs such as WinSCP, 
etc.). Your files will be immediately available on LAN at 
the URL ftp://192.168.1.200 or ftp://manasa. 


Practical Usage 

You may run X Window over the network with free pro- 
grams such as TightVNC (available also for Windows).All 
you need is to type vncserver:1 either in the root or guest 


2 THOUGHTS ON "HELLO WORLD! 


admin on February 4, 2013 at 11:34 am said: 

Posting comments in MaheshaBSD Server works immediately!!! To 
see how this works, follow the link on rare white cobra, which can be 
found at the the following link. Your comments are welcome! 


Reply | 


Figure 4. Leave comments without restrictions - a good way to 
inform colleagues how far you have progressed with your work 


by @ frp://192.168.1.200/ 


nd Reload stop 
jokmarks ESJAncient ETemp Eicomp Emap ESHeath Ee: search Ew E 








Index of ftp://192.168.1.200/ 
8 Up to higher level directory 


Name 
fe MaheshaBSOserverManual.pdf 


Figure 5. To have detailed information on how to use and connect to 
the MaheshaBSD’s FTP Server, see the documentation - it is available 
in the MaheshaBSD Server’s anonymous FTP server 
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account on the computer MaheshaBSD Server (or Ma- 
heshaBSD) is running on (it will act as a VNC server). The 
number 1 is the number of the display. If the VNC pass- 
word is not set up yet, vncserver will prompt you for it (you 
may change it anytime with vncpasswd). This way you 
may run phpMyAdmin (a free and open source tool written 
in PHP intended to handle the administration of MySQL 
with use of a Web browser) and many other GUI applica- 
tions remotely (in Microsoft Windows, Linux, Mac, etc.). 

To run TightVNC over the Internet (see the picture 
with Midnight Commander below), you must open the 
port 5900 (IP Forwarding); the last number is your dis- 
play number (+1, +2, etc.) — that is, to open MaheshaBSD 
Server to the world with display:2, the following port must 
be opened in your router: 5902. 


WordPress 
Log in as Admin with password manasa11. Go and oper- 
ate your own Intranet/Internet website. Be public. Have 


'y General Settings 
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Figure 6. /t is better to use WordPress with one fixed URL only, 
as changing the Site Address (URL) all the time your IP changes is 
complicated 
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Figure 7. /f you are a beginner, copy files with Midnight Commander 
(type mc in the command shell, then press F5 to copy a directory/file 
from the left panel to the right panel) 
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your own Internet shop, or the Intranet blog available to 
all employees. 

Before you open WordPress to the world (with IP For- 
warding) with MaheshaBSD Server, make sure you change 
your /etc/hosts in Unix, or C:\winnt\system32\drivers\ 
hosts in Windows (W2K, XP, 2003, Windows 7 & 8), other- 
wise your public WordPress will not be accessible. 

lf you decide to upgrade WordPress, keep wp-config. 
php in the /usr/local/www/wordpress directory, as it con- 
tains the database name used by WordPress. Upgrade/ 
expand anything you think is right for you. WordPress is 
configurable/expandable up to the maximum. 


Goals Of This Project 

To provide (users do not need any special knowledge of 
Unix) small businesses, institutions, households, etc., an 
immediate and smart FIP storage and a blog solution 
(WordPress) anywhere in the world. 

MaheshaBSD Server has an educational purpose, too. If 
you study operating systems, it is very valuable to see how 
FreeBSD works without installing it on your HD (depend- 
ing on various HD setups, installation sometimes requires 
repartitioning, a really painful process for many users). 

MaheshaBSD Server is a social project. Many people in 
the Third World need money for education and institutions 
often fail to provide the valuable infrastructure (software, 
paying for the development of software, etc.). 


Conclusion 

| thank http:/www.rootbsd.net for allowing me to distrib- 
ute MaheshaBSD and MaheshaBSD Server. The price for 
this software is 200/150/100/50/25 USD, but only for in- 
stitutions; MaheshaBSD Server is free for personal use. 
Institutions may choose any price, thus the software will 
be valuable also for institutions in very poor countries. 
Anybody can write and use it for free. Anybody is also 
welcome to donate. | plan to challenge people to donate 
money to all BSD projects. 


JURAJ SIPOS 

Juraj lives in Slovakia and he works in a library in an educa- 
tional institute. Some time in the past he was fortunate to trav- 
el around the world and he spent a bit of time in India and Aus- 
tralia. Juraj’s hobbies are computers, mostly Unix, but spiritual- 
ity too. His first published computer article was Xmodmap How- 
to (http://tldp.org/HOWTO/Intkeyb/). In addition to computers, 
he is very interested in Hinduism but not really the guru side of 
things, but more-so freedom and self-actualization. More at his 
website: http://www.freebsd.nfo.sk/ (FreeBSD), http://www.free- 
bsd.nfo.sk/maheshaeng.htm (MaheshaBSD). 
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The BSD Certification Group Inc. 
(BSDCG) is a non-profit organization 
committed to creating and 
maintaining a global certification 
standard for system administration 
on BSD based operating systems. 





@ WHAT CERTIFICATIONS ARE AVAILABLE? 


BSDA: Entry-level certification suited for candidates 
with a general Unix background and at least six months of 
experience with BSD systems. 


BSDP: Advanced certification for senior system administrators 
with at least three years of experience on BSD systems. 
Successful BSDP candidates are able to demonstrate 

strong to expert skills in BSD Unix system administration. 


@ WHERE CAN | GET CERTIFIED? 


We're pleased to announce that after 7 months of 
negotiations and the work required to make the exam 
available in a computer based format, that the BSDA 
exam is now available at several hundred testing centers 
around the world. Paper based BSDA exams cost $75 USD. 
Computer based BSDA exams cost $150 USD. The price of 
the BSDP exams are yet to be determined. 


Payments are made through our registration website: 
https://register.bsdcertification.org//register/payment 


@ WHERE CAN I GET MORE INFORMATION? 


More information and links to our mailing lists, LinkedIn 
groups, and Facebook group are available at our website: 
http://www.bsdcertification.org 


Registration for upcoming exam events is available at our 
registration website: 
https://register.bsdcertification.org//register/get-a-bsdcg-id 
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Reacting to Panic 





FreeBSD provides a very powerful mechanism for reporting 
dramatic error conditions that are known as kernel panic. 


What you will learn... 

« What is a system panic 

¢ How to configure the system to handle crash dumps 

¢ How to get information about what generated a panic 


kernel panic is when the system does not know how 
A: proceed any further, and instead of making the 
isks or the memory or any other component inco- 
sistent it raises a panic that is an explicit request for help. 
Fortunately, FreeBSD is a secure and stable operating 
system, and this means that panics are not frequent and, 
on the other hand, the system is able to start over automati- 
cally to provide service continuity. However, knowing what 
a panic is and how to handle information about it, as well 
as knowing when, in kernel or device driver development, 
it does make sense to issue a panic, which is a valuable 
knowledge for a system administrator. 


Let There Be Panic! 

The FreeBSD kernel provides a special function, panic (9), 
that immediately halts the system and prepares it for crash 
handling. The handling can be either entering the kernel 
debugger or dumping the memory content to one of the 
disks and reboot the machine. 

The panic (9) Call is restricted to kernel code, and, there- 
fore, no user program can generate a panic. On the other 
hand, any piece of kernel code, including loadable mod- 
ules, can do. This is also the reason for panic(9) to ex- 
ist: every time a piece of kernel code does not know how 
to proceed without tampering the whole machine (e.g., 
memory content, disk content, and so on), the system has 
to panic. 
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What you should know... 

« Basic Shell and Operating System concepts 
« Basic C programming language concepts 

« How to edit files and compile source code 


In the following, panics will be discussed in more details, 
showing to the readers when they can and when they are 
used within the kernel source code. In order to simulate a 
real situation, a kernel loadable module will be implemented. 
So, under specific conditions it will raise a panic. Finally, the 
inspection with the kernel debugger about what the panic 
was, will be shown. Readers are suggested to test exam- 
ples, code listings and in general the panic features on a test 
machine (for example, a virtual machine), in order to be able 
to apply their knowledge to production systems in the future. 


How Much Panic Can Be? 

The FreeBSD code base uses the panic(9) method in 
more than three thousand places in order to report very 
critical conditions. The idea is that each time the system 
does not know how to proceed, it generates a panic, so 
the system will, in the default configuration, reboot and 
start over. The choice is therefore that instead of becom- 
ing inconsystent, the system will try to start over again. 
This is a reasonable design choice, since panics should 
come from very strange situations, like an hardware de- 
vice that is not acting correctly. It is worth noting that pan- 
ics are not like assertions: panic (9) iS used to notify a run- 
time critical situation, while assertions are mainly used to 
signal a development error. To get an idea consider the 
start _init() routine (/sys/kern/init main.c) which is re- 
sponsible of starting the init process; what the function 
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does is essentially to search for a process to execute ina 
list of paths and to try to run it returning in the case of suc- 
cess. Therefore, as shown in Listing 1, if the exec (3) Call 
fails for each path entry, it means that the system has not 
found an init process to execute, and therefore it cannot 
continue. This is why the code of Listing 1 “launches” a 
panic if all the iterations of the for cycle fail. 


What Happens when the Operating System Goes to 

Panic? 

FreeBSD will not “scream” when entering a panic state, 
while it is possible some system administrators will do. 
As strange it sounds, the panic is still a quite “safe” condi- 
tion, because the system still knows what to do to produce 
an ordered shutdown. Due to a panic(9) Call, the system 
does the following: 


¢ starts an almost ordered (but forced) “shutdown” pro- 
cedure; 

¢ evaluates if the content of the volatile memory can be 
“dumped” on persistent storage (more on this later), and 
if possible stores the content of the memory on disk; 

¢ reboots itself. 


At the boot two main different scenarios can happen, the 
most common of which is to start the system in the usual 
way, as if an administrator manually turned the machine 
on. The second possibility is that the kernel enters a “de- 
bugging mode” (see the next section), and this is usually 
interesting for kernel and device drivers developers. 

Every time the system boots, an utility command named 
savecore (8) IS executed in order to inspect the storage 
and see if there is a memory dump; if a dump is found the 
dump is placed into a reachable place in the filesystem. To 
understand this better, it is important to understand what 
making a memory dump means. 

FreeBSD uses the swap space to keep a memory dump, 
that means that while the panic is in progress, the kernel 
will try to write the memory content to a swap area big 
enough to contain the dump (or will fail if no swap area can 
be found or can contain the dump). Since the swap area 
can be used during the boot itself, the usage of the swap 
area as persistent storage can be risky, and therefore the 
best the kernel can do is to place the dump at the end of 
the swap space, so that while booting the machine it will 
start using the initial swap space (if required) and chances 
are that swapped data will not overwrite the dump. More- 
over, since the recovery of the dump is done in the boot 
phase, there is a chance that the swap space will not be 
used entirely (for instance by some daemons), and so the 
dump can be restored. savecore(8) analyzes each swap 
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device configured searching for a consistent dump (foe 
example, a dump that has not been overwritten by other 
swapped data during the boot); if a dump is found, the 
savecore(8) COmmand moves such dump out of the in- 
secure swap space to a filesystem configurable directory. 
After that the boot continues as usual, the swap area can 
be used entirely and the dump can be inspected with a de- 
bugger. It is possible to tune the FreeBSD system speci- 
fying exactly where dumps have to be placed and which 
swap devices must be used to contain a dump: the spe- 
cial variables dumpdev and dumpdir placed in the rc. conf 
configuration file indicate respectively which swap device 
has to be used for dumps and to which file system place, 
the dump has to be written to. Listing 2 shows a configu- 
ration example whereas Table 1 summarizes possible val- 
ues for the above two variables in /etc/re.conf. 


Debugging a Panic 

On a development machine it could be useful to enter the 
debugger after a panic is issued, and FreeBSD allows the 
system administrator to configure the machine to do so. 
It does suffice to have the debugging options enabled in 
the current kernel and to enable the debug.trace_on_pan- 
ic syscall to have the system entering the debugger after 
a panic (see Figure 1). 





Listing 1. The mi_start() procedure 


Static Voud 
start init(void *dummy) 


{ 
for (path = init path; *path != ‘\0’; path = next) { 


if ((error = execve(td, &args)) == 0) { 
mtx unlock(&Giant) ; 
return; 
} 
if (error != ENOENT) 
printt(“exec.c." ss -ernror 
sown > Vint) (mext = palin), 
Death, error); 
} 
PeInEE (ie Mou hOUNG Ne stings le per alieearh)., 
evsuhen GM ave eal onsline, 2) 


} 


Listing 2. An example of configuration for the dump tunables 


dumpdev=”"/dev/ad0slb” 


dumpdir="/usr/crash” 
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Generating Panics On The Fly 
The FreeBSD system allows user administrators to manu- 
ally crash a system via the debug.kdb.panic sysctl; it does 
suffice to set the above to true (value 7) to see the system 
having a panic and rebooting: 


# sysctl debug.kdb.panic=1 


While generating panics, using the above sysctl can be a 
good solution, it does provide the background only for test- 
ing the panic-handling configuration of the system and does 
not allow very much testing and usage of the kernel de- 
bugger to discover a few information about what generated 
the panic itself. Therefore, in the following, a specific kernel 
module will be implemented to exercise the panic handling. 


The panicModule 

The panic (9) special function is not visible to userland pro- 
grams, and can be called only from within the kernel con- 
text; for this reason in order to use such function to gener- 
ate on-demand panics it is required to implement a kernel 
module. An exhausting explanation of kernel module pro- 
gramming is out of the scope of this article, and the mod- 
ule presented here has to be intended only for didactic pur- 
poses. This means the module does not strictly follow the 
kernel code style (9) and does not apply optimizations. The 
module has been implemented and run on FreeBSD i386 
versions 8.2 and 9.0-RELEASE, but should work without 
any changes on other releases as well. The logic behind the 
module proposed here is quite simple: the module will wrap 
two file-level system calls, open and mkdir, and will gener- 
ate a panic if the path the above system calls are going to 
be executed contains specific words (for example, “crash’). 
Given a list of “bad” words defined as {“panic”, “crash”, “bs- 
dmag’”} the system will panic if: 


Table 1. dumpdev and dumpdir possible values 


Variable Value Meaning 


NO Disables the crash dump mechanism at all. 


¢ a directory which name (case sensitive) contains at 
least one of the bad words; 

¢ a file which name (case sensitive) contains at least one 
of the bad words and the file is going to be written (for 
example, open in read-write or write-only mode). 


While this can seem too complicated for testing panics, it 
simulates a real module: if the execution context is good 
(for example, names are clean) the module does its stuff 
while, if the context is bad (for example, names contain 
bad words) the module does not know how to proceed 
and therefore gives up generating a panic. 

The module event handler is shown in Listing 3. 
The module stores the list of bad words, defined as 
panic module argv t) In a global variable named 
moduleInitializationData; such variable will simplify the 
analysis from behind a system call wrapper. Once the 
module is loaded (MVOD_LOAD), the initialization argu- 
ment of the module is stored in the global variable and 
the sysent entries for the mkdir and open system calls are 
changed so that their implementations point to the “pani- 
cable” versions. It is worth noting that the module stores 
in internal variables the old system calls implementation 
in order to put back in place the original (i.e., not panic- 
ing) system calls once it is deactivated (WOD_UNLOAD). 

Listing 4 shows the implementation of the mkdir and 
open wrapping implementations that can generate pan- 
ICS (respectively panicable mkdir and panicable open). 
The flow is really similar on both functions panicable_mk- 
dir and panicable open: the path and the permissions are 
extracted from the system call arguments and checked 
against each entry in the list of the bad words (accessed 
via the moduleInitializationData global variable); in the 
case a match is found the flag shouldPanic is set. At the 
end of both methods if the variable shouldpanic Is seta 





dumpdev | AUTO 


Uses the first swap device listed in /etc/fstab. 





<swap device> 


Uses the specified swap device for the dumping mechanism. 





NO Do not extract a dump from a dumpdev. 





dumpdir | <none> Uses the default location /usr/crash. 














<directory> 


Place a found crash from dumpdev into the specified directory. 








Savecore: reboot after 


panic: Generating a panic from mkdir system callt 


Jul 31 09:08:36 bsdmag savecore: reboot after panic: Generating a panic from mkd 


ir system callt 
Savecore: writing core to vmcore.4 





Figure 1. Entering the debugger from the system console after a panic 
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Jriting crash summary to /usr/crash/core.txt.4. 
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Listing 3. The panic module event handler 


Staure int 
panicEventHandler ( 
struct module *st module, ee 
pointer to the module struct */ 
int event, /* the 
event to process for this module */ 
void* argv PS 


custom arguments for the module */ 


else{ 
/* set the module arguments to null */ 


moduleInitializationData = NULL; 


/* insert the panicable system calls in the system 
table */ 

uprintf( “\nInstalling a panicable mkdir system 
Caluieee ea = 


eysioime|| Svs uillkechiie \|asy ieedlIl = {sy eI te 4) 





{ panicable mkdir; 
uprintf( “\nInstalling a panicable open system 
/* Define a variable to handle the exit status (0 on Cade ae 
silececaa 7 SVSeSUe | oto ODen Wes yecalll = (sy call € *) 
int error = 0; Pantcable vopen,; 
x 
* Define clean (unmodified) system call handlers. break; 
These will be used 
* to revert the original system calls when the /* the module is being unloaded */ 
Mo Clie ews SUM acl cy, case MOD UNLOAD: 
ah 
syucallyt clean mkduresystemecall == sysent)| (ovo cmkdar /* print a message in the logs */ 
(es eco: uprintf( “\nUnloading (%d) the module %s ...\n”, 
Syeca lier. clean ocennsyscemyeal le — sy seni o> open Se mie, a einen ji 
esvacellt, 
/* restore the original system call */ 
/* which kind of event is happening for this module? */ uprintf( “\nRestoring the original system 
switch( event ){ Galle tae: )ie 
SV Semele 4S amichiaa | sve alle = elo anemic Ss yo vem 
/* the module is being loaded */ Galil; 
case MOD LOAD: Svecme|| SoS joe |) esi eel ll S veliSein opis syreicci 
/* print a message in the logs */ Cail ils 
Uprintt( “\nloeding (2d) Ene module ss) 2. .\n", 
Sueie,, iy iene )) 2 break; 
/* get the arguments for the module */ default: 
if( argv != NULL ){ /* if here the event is not supported by this module */ 
moduleInitializationData = (panic module argv t*) uprintf( “\nEvent/Command %d not supported by module 
argv; s\n”, event, my name ); 
error = EOPNOTSUPP; 
for( int 1 = 0; i < modulelInitializationData- break; 
>count; it+ ){ } 
uprintf( “\nWill generate a panic on filesystem 
name containing [%s]”, 
moduleInitializationData->names[i] ); / ee COne 7) 
} return error; 
} /* end of argument for the module */ 
16 BSD 03/2013 








Listing 4. The mkdir and open panicable wrapper implementations 


Ite panvceoletiidta( ssenuen tnveads = uimead, 
SULUCE Mk¢dir args. *vap 


) 


/* a flag to indicate if should generate a panic or not 
as 


inh chew le Banniie = 20: 


/* check if the path for the mkdir call 
is contained into one of the panic paths defined 
at the time of module load */ 
for ( ane i7= 0): 
moduleInitializationData != NULL && i < 
moduleInitializationData->count; 
it+ ){ 
if ({stests( uap--pakn, moduleinitrelirzatronData— 
>names[ i] ) != NULL ){ 
/* the path contains a panicying word! */ 
shouldPanic = 1; 
uprintf( “\nThe path [%s] will generate a panic, 
HE CON EAIMS Wanwe word (os ((imdex <c) 
Nai 
Uap->parh, 
moduleInitializationData->names[ i ], 
i); 


break; 


J/* should we call the standard mkdir or panic? ~/ 
if( ! shouldPanic ){ 
i VO Gaara gular sc allie 77 
return cys mkeie( eEhnvead) Uap), 
} 
else{ 
panic( “Generating a panic from mkdir system call!” 
); 
return shouldPanic; /* should never get here, just 


to keep quite the compiler */ 


Iie pemecakemoeen (Ge jeruces ekmeeacd Puhweady 
SUCUGU, Open ange “Walp 


) 


/* a flag to indicate if should generate a panic or not 


ay) 





ies home? cies — 0) 


/* check if the path for the open system call 
is contained into one of the panic paths defined 
at the time of module load, and if so check also 
the 
opening flags to see if the file 1s going to be ope 
ned 
for write 
ae 
for(int 1. 0; 
moduleInitializationData != NULL && i < 
Modell Zee ronDaba--counre; 
it+ ){ 
Lf ("strstr (ap->parn, moduleimitialazarionDara- 
>names[ i] ) != NULL 
&& ( (uap->flags & O WRONLY) == O WRONLY 
|| (uap->flags & O RDWR) == O RDWR ) 
Hl 
/* the path contains a panicying word, and the file 
is being opened for writing! */ 
shouldPanrve = 1; 
uprintf( “\nThe path [%s] will generate a panic, 
ie ACOntainis patine word {os ))( iidex 
Cie ein atlacereca. in 
Uh =e Oene |, 
moduleInitializationData->names[ i J], 
1, 
uap->flags) ; 


break; 


/* should we call the standard open or panic? */ 
1£( ' Sshouldraniie ) { 
7 Soi de. a cegqular call *7 
betucn S75 epen( thread, uae.) 
} 
else{ 
panic( “Generating a panic from open system call!” 
); 
return shouldPanic; /* should never get here, just 


to keep quite the compiler */ 
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Listing 5. The definition of the module 





Statice chav my mame |] =~ PreesoDPanicModute”; 
Stabic panic module argv te argv = { 
3, 


 OetEe sera on osama + 
le y T 


He 


panic module argv t* moduleInitializationData = NULL; 


Statue moduledata te pante modulerdemnittron =" | 


my name, /* module unique name */ 
panicEventHandler, /* event handler */ 
& argv /* module arguments */ 
}; 
DECLARE MODULE( panicModuleBin, /* binary file name */ 
panne piodule cdeciinacnony, ) Snecma lic epise a maml =a, 
Sl esUE DRIVERS, /* subsystem */ 


SI_ ORDER MIDDLE / ine ice nom momder a) 


ye 


Listing 6. Loading the kernel module into the system 


# kldload ./panicModuleBin.ko 


Loading (0) the module FreeBSDPanicModule 

Will generate a panic on filesystem name containing [panic] 
Will generate a panic on filesystem name containing [crash] 
Will generate a panic on filesystem name containing [bsdmag] 


Mints worl lenme eaaiOamne alolko miieclisteggs yc eommec mlanere 


installing a panicable open System call =. 





# kldstat 
Id Refs Address Size Name 
il 20 Oxc0400000 bd98b4 kernel 
Z PP Oxc0tdad00) 23820 Sp lasiapex ake 
2 1 OxcOfdd000 6434 vesa.ko 
~ EO xeZc ms 000M O00 ZS sO 
o I Oxe2 tr sZ000" S000 opensolaris.ko 
6 1 0xc4743000 3000 daemon saver. ko 
a Ll OxeZ ccd 00 2000 panicModuleBin.ko 
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panic(9) Call is executed and the system crashes, oth- 
erwise the original system call is invoked. For the sake 
of clarity, Listing 5 shows the glue code that defines the 
module, as well as the initialization data that contains the 
list of words to match to produce a panic. 

The Makefile for this module, which code is split among 
different source files, is the following: 


KMOD= panicModuleBin 
SRCS= panicModule.c panicModuleSyscalls.c 


-include <bsd.kmod.mk> 


and will produce a panicModuleBin.ko kernel module 
(KLD) that can be loaded with the kidioada(8) Command 
as shown in Listing 6. It is interesting to note that the mod- 
ule informs the administrator of the bad words on which it 
will generate a crash. In order to generate a panic it does 
suffice to create a directory or write into a file with a bad- 
word name as for instance (it does not matter the user 
identity since the wrapping system calls do not check it): 


mkdir my crash directory 


and the system will immediately crash showing messag- 
es similar to those in Figure 2 on the console. It is worth 
mentioning that the console shows the exact message 
the module has passed as argument to the panic(9) 
function. It is also worth noting, as shown in Figure 3, 


i 


Figure 2. Console messages immediately after a panic and before the 
system reboots 





Figure 3. Kernel messages indicating the saving of a panic dump 
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Reacting to Panic 


that while booting the system detected a crash-dump 
and is going to restore it on an accessible file system for 
further inspection. 

In order to use the panic module in a more comfortable 
way, it is worth installing it in the system so that the mod- 
ule can be found by the loader and the kernel; in particular 
issuing a make install will copy the kernel module object 
into the default path for modules /boot/kernel; it is impor- 
tant to export the symbols for debugging in order to better 
see what is happening: 


> make DEBUG FLAGS=-g3 

# make DEBUG FLAGS=-g3 install 

install -o root -g wheel -m 555 
/boot/kernel 


panicModuleBin.ko 
install -o root -g wheel -m 555 panicModuleBin. 

ko.symbols /boot/kernel 
kldxref /boot/kernel 


The Panic Configuration 
Now, when it is possible to generate a panic “on-demand”, 
simulating what a real complex kernel module would do, 
it is time to have a look at how FreeBSD handles panics. 
As already explained, when the system crash, by means 
of the panic 9) call, FreeBSD dumps the content of the 
whole memory to a disk, in default to the end of the first 
swap partition configured in the system. When the sys- 
tem is booted again, the swap partition is checked to see 
if it contains a previously recordeded dump, and is such 
case a specific program called savecore (8s) handles the 
dump and moves it from the swap to a file-system acces- 
sible space, that is to a directory in the system (in default 
/var/crash). AS already seen, it is possible to configure 
either the device/swap partition to use and the final desti- 
nation of the dump using respectively the variables dump- 
dev and dumpdir in /etc/rc.conf, aS Shown in Listing 7. 
Once a panic is generated, for instance loading the pan- 
ic module of the previous section and creating a “bad” di- 
rectory is as follows: 


# kldload panicModuleBin.ko 


# mkdir go crash now 


the system will automatically reboot and a crash dump will 
be present in the dumpdir location as shown in Listing 8. 
As readers can see from Listing 8, there are few numbered 
files in the dumpdir location, so more crashes can coexist in 
the same place (in the above example the number 0 means 
no crash have been produced before). The vmcore.0 file is 
the biggest one and represents the image of the memory 
at the time the machine crashed. The info.0 file is a text file 
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Listing 7. A sample configuration of /etc/rc.conf for handling crash 
dumps 


# grep dump /etc/rc.conf 

dumpdev=”"/dev/ad0slb” 

dumpdir="/usr/crash” 

# Swapinfo 

Ime locks Deed 
610256 0 


Device 


/dev/ad0slb 


Avail Capacity 
610256 0% 


Listing 8. Acrash dump files after a reboot 


7 lee line lier eras) 


-rw-r--r-- 1 root wheel Yelle oe 4 belumas 
-rw------- 1 root wheel hey hin aoe hol) Tsliaisiena) 
-rw------- 1 root wheel o7M dul 23 15:40 vmcore. 0 


# more /usr/crash/info.0 
Dump header from device /dev/ad0slb 
Architecture: 1386 
Architecture Version: 2 
Dump Length: 101822464B (97 MB) 
Blocksuze: 312 
DUuMpet ne ss Momuitiee 2s esos Se Oe 
Hostname: bsdmag 
Magic: FreeBSD Kernel Dump 
Version String: FreeBSD 8.2-RELEASE #2: Mon Jul 9 
iz 592 SoeUre 7202 
root@bsdmag: /usr/obj/usr/src/sys/GENERIC.colors 
Panic String: Generating a panic from mkdir system 
ca lt 
DUM e Ee etaieye eGo Ue lkZ 
Bounds: —0 
DUMPS oeabUs: «dood 
7 Gadi sh) Chachi lheunes 


il 


Listing 9. Starting the kgdb debugger 


# kgdb -d /usr/crash/ -n 0 kernel.debug 

GNU gdb 6.1.1 [FreeBSD] 

Copyright 2004 Free Software Foundation, Inc. 

GDB is free software, covered by the GNU General Public 
License, and you are 

welcome to change it and/or distribute copies of it 
under certain conditions. 

Type “show copying” to see the conditions. 

There 1s absolutely no warranty for GDB. 


Type “show 


Wakraniy “LOrsdecraris: 





This GDB was configured as “i1386-marcel-freebsd”... 


Unread portion of the kernel message buffer: 


panic: Generating a panic from mkdir system call! 


cpuid = 0 

KES ew Stack backurace, 

#0 Oxc08e0d07 at kdb backtrace+0x47 
Te OxeOci ide] wath spamnietUscll 7 

#2 Oxc4allc39 at panicable mkdir+0x89 
#3 Oxc08eca39 at syscallentert+0x329 
#4 OxcObe4el4 at syscall+0x34 

7) UxcObebr7l Tat aimr0 x30 syscallt0x71 
Uptime: 21m28s 

Ehysical memory: 239° ME 

Diimiouenc Oy SMB 82 ogo 0) 34 segs 7 


Reading symbols from /boot/kernel/splash pcx.ko...done. 

Loaded symbols for /boot/kernel/splash_pcx.ko 

Reading symbols from /boot/kernel/vesa.ko...done. 

Loaded symbols for /boot/kernel/vesa.ko 

Reading symbols from /boot/kernel/zfs.ko...done. 

Loaded symbols for /boot/kernel/zfs.ko 

Reading symbols from /boot/kernel/opensolaris.ko...done. 

Loaded symbols for /boot/kernel/opensolaris.ko 

Reading symbols from /boot/kernel/daemon_saver.ko...done. 

Loaded symbols for /boot/kernel/daemon_saver.ko 

Reading symbols from /boot/kernel/panicModuleBin.ko... 
Reading symbols from /boot/kernel/ 
panicModuleBin.ko.symbols...done. 

cone 

Loaded symbols for /boot/kernel/panicModuleBin.ko 

#0 doadump () at pcpu.h:231 

Zi Deacon Mev ccte 10, 0 See ae aed), 

(kgdb) 
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that contains a summary of the crash generation, such as 
the time of dump, the kernel ident string, the message of the 
panic(9) Call (it is worth noting that it is the same string as 
the one in the panic module) and the size of the dump (for 
example, the size of the file vmcore.0). The bounds file sim- 
ply contains the name of the next-to-dump set of files: a val- 
ue of 1 (as of Listing 8) means that the last dump was num- 
ber 0 and that the next dump-on-crash will be numbered 1. 

It is worth noting that the files are owned by the system 
superuser (root) and cannot be read by other users: in 
fact, the memory dump contains the exact dump of each 
byte and therefore includes potentially reserved informa- 
tion. Making the file not world-readable is a little security 
measure to allow only root to inspect such files. 


Inspecting a Crash Dump 

In order to inspect a crash dump the kgdb(1) utility is re- 
quired. kgdb (1) is the kernel debugger, a modified version 
of the GNU gdb debugger specialized for analyzing a Free- 
BSD kernel data. The debugger is interactive and requires 
to know the core file to inspect or, alternatively, the directory 
where the crash dumps are and the number of the dump to 
inspect. As shown in Listing 9, once started, the debugger 
inspects the core file (vmcore.0) and reads the symbols off 
all modules, including the panic module one. Having loaded 
the dump data, is now possible to inspect what happened 
when the system crashed. The symbols for the module are 
automatically loaded if the module is in the path, otherwise 
it is possible to use the add-symbol-file command of khd (8) 
to add a symbol file from an arbitray path. 

It is worth noting that if the kernel has been compiled 
with debugging symbols, the kgdb 8s) tool can extract 
more accurate and useful information about the dump. 
In particular, the kernel presents a kernel.debug directory 
with all the symbols in the compilation directory for the 
configuration (usually /usr/obj/usr/srce/sys/<KERNCONF>); 
it does suffice to start the kgdb(8) Command specifying 
the directory that contains the kernel symbols as follows: 


cd /usr/obj/usr/src/sys/<KERNCONF> 
kgdb -d /usr/crash -n 0 kernel.debug 


In the following it is assumed that the kgab (8) debugger is 
started using the kernel symbols; please note that the di- 
rectory to use is specified in the info.0 file in the kernel ident 
string; it is also possible to create a simple shell wrapper that 
will start kgdb (8) with the correct options as shown in Box 2. 
The first thing that the kgdb 8) utility prints is a kind of 
“dump header” that corresponds to the message printed 
out on the console while the system was halting (see Fig- 
ure 1): the message that has been used aS panic(9) ar- 
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gument as well as a memory size that will be dumped and 
the stack trace related to the crash. 


Please note 

That the data printed in Figure 1 is not effectively the 
same as the two panics generated are different. However, 
the concept is that the printed message from kgdb (9) Is 
the same as printed out on the console when the panic 
has been generated. 


The first step is getting an idea of what were the last ex- 
ecuted instructions, and this can be done inspecting the 
stack trace, that is unrolling the method calls that ter- 
minated with crashing the system. The backtrace com- 
mand can be used to inspect the stack trace as shown in 
Listing 10. The stack trace is printed out in reverse order, 
that is the top of the stack represent the most recent call. 
Therefore, as shown in Listing 10, the call to panic(9) 
(line #2) came from a call tO panicable_mkdir() (line 
#3) which in turn was made by entering a system call via 
syscall enter() (line #4); the last executed instruction 
WaS doadump() (line #0) that halted the system writing 
down to disk a dump of the whole memory content. 

From the above simple information it is possible to geta 
first idea of what happened: the system crashed due to a 
voluntary panic (9) Call, that happened into the panicable | 
mkdir method call. In particular the latter is a system call, 
since it is on the direct stack of syscall. 

It is possible to get much more information inspecting 
what happened within the sycall/: the kgdb 8) allows the 
selection of a specific frame on the stack and the inspec- 
tion of the local variables, as shown in Listing 11. The first 
step requires the selection of the frame number, that is the 
number reported in the back trace (5 in this example), and 
then using the /ist command it is possible to see the code 
executing in such frame: the last line, the 1061, is the line 
that caused this frame to make another call, and corre- 
sponds to the syscallenter method call. Using the info lo- 
cals command it is possible to get the dump of the locals 
variables, that is variable defined into the currently select- 
ed stack frame. One variable in particular, sa, is important: 
it is the argument of the actual system call and it provides 
the data that caused the system call to crash the system. 
The first thing to note is the code attribute, that has a val- 
ue of 136 corresponding to a mkdir system call. There- 
fore, even without knowing the names or the instructions 
that caused the panic, it is now possible to state that the 
system was executing a mkdir system call. 

Inspecting the stack frame related to the panicable_mk- 
dir reveals also more information, and the inspection could 
have started from there, but it is worth noting that such level 
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of information could not be always available to the adminis- 
trator, and this is the reason why the analysis started from 
the previous frame. As shown in Listing 12, the line 48, that 
is the last line of code on such executed frame, is the explic- 


it call to the panic 9) call. Therefore, it is possible to state 
that this is the point where the system voluntary crashed. 
As a quick summary, as shown above, from a crash dump it 
is possible to get the information about the time and size of 





Listing 10. Examining the stack trace that lead to acrash 


(kgdb) backtrace 
#0 doadump () at pcpu.h:231 
tl OxcOsblocs im boot (howro—260) “at /uls“/sne/sys/ kern/kerm shucdown.c:419 
#2 OQOxc08ble00 in panic (fmt=Variable “fmt” is not available. 
) at /usr/src/sys/kern/kern_ shutdown.c:592 
io OU xe2 DUrd oo ine pan beab eomkdir W(varcad-Variables entcad SES OL eavallabile. 
) at panicModuleSyscalls.c:48 
#4 Oxc08eca39 in syscallenter (td=0xc434b870, sa=0xcca2dce4) 
aibs/Wet/sre/ sys/kern/ subi ecapsce: slo 
#5 OxcObe4el4 in syscall (frame=0xcca2dd28) at /usr/src/sys/i386/i386/trap.c:1061 
(oO WxcObCbi7! im xincOxsUe syscall () wan User, sre) sys, 1366/1306) exceprlon. s+ 704 
To ORO O00 02s) ine? 2 7) 


Listing 11. Exploring the syscall stack frame 

(kgdb) frame 5 

#5 OxcObe4el4 in syscall (frame=0xcca27d28) at /usr/src/sys/i386/i386/trap.c:1061 
1061 error = syscallenter(td, &sa); 


(ced) alive 


RONG Chg frm cage = kvaNc o> rm scilage, 
BOSSY 

EOS td = curthread; 

LOS? td-stdaavane = hreane, 

1060 

1061 error = syscallenter(td, &sa); 
1062 

Ge /* 

1064 * Traced syscall. 

LOGS */ 





(kgdb) info locals 
td = (struct thread *) 0xc439d000 
Sa = {code = 136, scallop = Oxc0dse3c0, args = {-l07/9408738, Sill, 0, -1077942756, 0, 0, 0, 
-1002844160}, narg = 2} 
CEG sen eillagse, = 967 
error = -1002844160 
ksi = {ksi dink = {tqe mext = OxceaZ/dzZ@, tEqe prev = Oxc0dici3s0}, ksi vinio = { 
SRESUCMO = 9s, Siperrne, = 990780536, surcode = 07 ssl pid = 93, .sturd = 4, 
Si vstatus = 12 ssivaddr — 0x23 lociZc, Ss. value — {eivallint = 6/2580652, 
sivalepre = UxZ28iec42c, sigvalvine = 6/25980652, sigvallprr = 0x281oc4Ze}, 
Sheasoie— (yhaule.— (otrapne — 519) etinen = {peinesid = 319, 
Devermun = 661760420) mesgqq = { mgd = 519}, poll) = { band = 519), 
Despare = {isparela ee — olo, epare2 Se = {=s3el/cer20 =0G4so9els, =10uzZs44o0, 
—I907 60586 5 -ool O64 0 —0G4 365552, ~lL007Ze44 Ico} }}) ) Ksigtlags — 1002344160, 


kstesigg = Uxecaz)cec} 
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the dump (for example, from the info.0 file), and thru the de- 
bugger kgdb (8) it was possible to find out that the last sys- 
tem call executed was a mkdir (number 136) and that such 
syscall, that resolved into a panicable_mkdir autonomusly 
generated a dump calling panic (9) at line 48. 

It is now possible, if the compilation allows to (see Box 
1), to inspect also the local variables for the system call 
stack frame; the procedure is similar to that presented be- 
fore and is shown in Listing 13. Unluckily, while it is quite 
simple to see locals, heap based variables cannot be in- 
spected since their value could be in user space and not 
in the kernel address range. This is the case of the uap- 
>path variable, which is a pointer to the memory address 
Oxbfbfee16 that is under the 3GB memory limit (32 bit ar- 
chitecture) oxco00000 (address over this limit are limited 
to kernel usage). In this case it is possible to do a change 
to the panicable mkdir system call in order to either copy 
the variable on the stack or into a kernel memory address. 
Listing 14 shows both the approaches. Listing 15 shows 
the result of such change: the string go crash now IS NOW 
correctly reported in both the local stack variable and the 
kernel memory pointer. Therefore, in this lucky case it is 
possible to state that the crash was generated by a mkdir 
system call with go crash now aS main argument. 

In the unlucky case there is no way to access the content 
of the uap->path variable, a more deep inspection has to 
be done. Having the source code and the debugging sym- 
bols is very helpful: in fact the administrator/developer can 
inspect more code, for instance what happened before the 
generation of the panic, and try to understand what hap- 
pened and what brought the system to panic. So far, the 
administrator knows that a mkdir system call has been in- 
voked, and that the shouldPnanic variable has been set, so 
to generate a panic (see Listing 12). But where was such 
variable set? The /ist command can help: the debugger can 
show previous lines on the selected frame, so that giving 
list -10 will print the code of the system call implementation 
10 lines backward at time. Therefore, the code that hap- 
pens is that of Listing 16, that reveals that the shouldPanic 
flag is set depending on the values contained in a global 
variable moduleInitializationData. It is then possible to 
manually inspect such variable to get more clues; as shown 
in Listing 16 the dump reveals all the bad words used to 
panic the system. Therefore, having the source code in- 
formation in this case helps to the point that a mkdir sys- 
tem call executed with one of the bad words contained in 
moduleInitializationData Crashed the system. 


The Dumping Mechanism: a Few Low Level Details 


It is interesting to navigate the operating system source 
code in order to see what is effectively done when a dump 
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needs to be written to persistent storage. In the following, 
a quick explanation of the main routines involved in the 
dump mechanism is presented. 


The dump and shutdown sequence 

The kern_shutdown.c file defines the doadump() function 
that is responsible for starting the dump of the memory. 
The doadump () function essentially checks that a dump de- 
vice is configured (otherwise it simply returns) and then 
performs the saving of the context registers and starts the 
real workhorse for dumping, that is the dumpsys(..) func- 
tion as shown in Listing 17. The dumper object is defined 
aS dumper info Struct and one of its most important field 
is the dumper one, that is a pointer to the function that is 
going to do the real dump of the data. 

The dumpsys(..) function is a low level function and is 
therefore machine dependent; as an example consider the 
implementation for the Intel x86 architecture contained in 
sys/x86/x86/dump_machdep.c and which snippet is shown 
in Listing 17. A detailed explanation of such low level func- 
tion is out of the scope of this article, so a quick walk through 
will be presented. Initially the dumpsys(..) function checks if 
the media pointed by the di variable contains enough space 
to handle the whole dump (which size is expressed by the 
dumpsize variable) and the headers of the dump. Suppos- 
ing the media has enough space, the dumplo variable is 
initialized with the location offset of where the dump must 
be placed. Please note that the dump position is computed 
starting from the end of the media and coming to its start, so 
that the dump will result at the end of the media. This is due 
to the fact that the media is effectively a swap used storage, 
and since the system will require swap space to boot, plac- 
ing the dump at the beginning of the swap will make the boot 
process to overwrite it. Placing the dump at the end of the 
swap space give more chances that the boot process will 
not consume so much swap area to overwrite the data that 
is at the end. The remaining of the dumpsys(..) performs 
a set of dump write ..) Calls to write the header and the 
whole memory content. The call to the foreach chunk/(..) 
method performs the actual volatile memory dump. The 
foreach chunk(..) In turns call the cb dumpdata(..) (see 
Listing 17) that essentially does the following: 


¢* computes the number of pages to dump and store it 
In pgs variable; 

* computes how many pages can be dumped in a sin- 
gle I/O operation on the media (maxdumppgs); 

¢ executes a while loop on the number of pages to 
dump pgs and for each iteration computes the num- 
ber of pages that will be dumped (chunk) and the size 
of the dump (sz); 
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Listing 12. /nformation from the panicable_mkdir stack frame 


(kgdb) frame 3 
Wo OxXe7DUECSo in panicable miki \(ctivessd—Var rable 
wuhiecead “1s noc available. 


) at panieModullesSyscalls c:48 





48 panic( “Generating a panic from mkdir system 
Cade ye 
(kodb)- list 








43 if( ! shouldPanic ){ 

44 /* ok, do a regular call */ 

A5 (GS eee) oe wel ints (wale ciel pam cl ome), 

46 } 

4’] elsef{ 

48 panic( “Generating a panic from mkdir system 
Celslats-\).; 

49 return shouldPanic; /* should never get here, 
just to keep quite the compiler */ 

50 } 


Listing 13. /nspecting local variables for the panicable_mkdir 
system call 


(kgcb) “inte lecale 

shouladPanire = I 

(kgdb) set print pretty 

(kgdb) print *uap 

sl = { 
Pabiek, = Uxcdbascec ~ (O2Z6NN 001 | 
path = Oxbfbfeel6o <Address Oxbfbfeel6 out of bounds>, 
path r= Uxcdbaser0 001", 
mode 1 = Oxcdba3cf0 “O\001", 
mode = 511, 
modes = Oxcdbasci4 ~~ 

} 

(kgdb) print * Oxbfbfeel6 


Cannot access memory at address Oxbfbfeelo 


Listing 14. The first part of the panicable_mkdir implementation 
that can copy the path into a local variable or a kernel memory 
address 


#ifdef MALLOC ARGS_ 
static MALLOC DEFINE( M PANIC MEMORY, “mkdir-path”, 


“Argument to the mkdir system call” ); 


#endif // MALLOC ARGS _ 


impel Hea Ole mmiNche @eteeie te eimea ce melted cl, 


SE BUC ts Ike Liga gel = wale 


) 


/* a flag to indicate if should generate a panic or not */ 


int shouldPanic = 0; 
#ifdef LOCAL PATH 
char localParn| 255); 


/* zero fill the memory */ 
fOE\ alee gO ers 255, ae) 
hocalPariih @ale——) Ue; 


/* copy the mkdir path to a local variable, so it will 
be available 
when using the debugger */ 
Strep, ( localeath, uep-sparm )-; 
#endif /* LOCAL PATH */ 
#ifdef MALLOC ARGS_ 
ChagePauie —amalloe ysier Ken (tap seach) et lee 
PANIC MEMORY, M NOWAIT ); 


// copy the uap into the kernel version 
eCopyinstr( uap—-s>path, kPath, strilen( uap-syoath }, NULL ); 
#endif // MALLOC ARGS _ 


Listing 15. /nspecting the local copy of the path. 


(cocb) into tocals 

(kgdb) into Vocals 

shouldPanic = 1 

localPath = “go crash now”, ‘\0’ <repeats 242 times> 

localPathPointer = Oxbfbfeel6 <Address Oxbfbfeel6 out of 
bounds> 

kPath = 0xc43e5160 “go crash now” 


Listing 16. /nspecting the modulelnitializationData global variable 
(kgdb) print moduleInitializationData 

$17 = (panic module argv _t *) 0xc4a3606c 

(kgdb) print *moduleInitializationData 


$18 = { 

CcOuUnE = 3, 

names = 0xc4a36070 
} 


(kgdb) print (*moduleInitializationData) .names[0] 
$19 = 0xc4a34df£5 “panic” 

(kgdb) print (*moduleInitializationData) .names[1] 
$20 = Oxc4a34dfb “crash” 

(kgdb) print (*moduleInitializationData) .names [2] 
$21 = 0xc4a34e01 “bsdmag” 
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Listing 17. The doadump() kernel function 


Static voud 
doadump (void) 


{ 
if (dumper.dumper == NULL) { 


/* Dump 


printf(“Cannot dump. Device not defined 


or unavailable. \n”); 


return; 


savectx (&dumppcb) ; 

Gung eid: —scugrhitead Fred srnd, 
Gumeisag a 

dumpsys (&dumper) ; 


Cunomng==; 


Starle SEMUCE dumperimro dumper; 


// feos )s/eye/conn.a 


SErMUCE CUMperimie: { 


Startle 


ComcUinpcdatalst Ble mda pae dp mult Ss ecnia, VOCs Aciiag)) 


{ 


dumper t *dumper; /< Dumping shiinceven </ 

void Aenea? /* Private parts. */ 

Up ie olecksirze,; /* size ef block in 
bytes. */ 

Ue eEmis max1losize; Je Maes 17s liowea. for 


ameindiy vue 1/0. *7 


off t mediaoffset; /* Initial offset in 
bytes. */ 

Opi ye lcd tasuze, /* Space available in 
bytes. */ 


ie 


Listing 18. An extract of the dumpsys(..) function for the x86 


architecture 


void 


dumpsys(struct dumperinfo *di) 


dumplo = di->mediaoffset + di->mediasize - 
dumpsize; 


dumplo -= sizeof(kdh) * 2; 


memory chunks (updates dumplo) */ 
SrnOm = rOreach ohunk (che dunpdara, di); 
if (error <0) 

GCrOmeaHel: 
dumpuwrite (dil, NULL, 07 0, 0); 
printf(“\nDump complete\n”); 
return; 
IE EG, 


struct dumperinfo *di = (struct dumperinfo*) arg; 
wMppaddt (ea, pay; 
y Onna = cle, 


UMECt Se pgs; 


pgs = mdp->md_size / PAGE SIZE; 
pa = mdp->md_ start; 
maxdumppgs = min(di->maxiosize / PAGE SIZE, 
MAXDUMPPGS) ; 
if (maxdumppgs == 0) /* seatbelt */ 
maxdumppgs = 1; 


while (pgs) { 
chunk = pgs; 
if (chunk > maxdumppgs) 
chunk = maxdumppgs; 
sz = chunk << PAGE SHIFT; 


counter += SZ; 


tor Vy = Oe lr Chunk? sea) 








{ a= pa + i * PAGE SIZE; 
lise po els Sinaia 7 Va = pilap. kenter — 
UlDbO4 te aumps ze, temporary (PEune page (a); 2); 
Gif se hdugap,; } 
Sula i laches error = dunpowrite(di, va, OU, dune ie, 
int error; So 
if (error) 
li (di->mediasize < SIZHOr METADATA + dumpsize + break; 
sizeof(kdh) * 2) { dumplo += sz; 
error = ENOSPC; pgs -= chunk; 
COO welll pa += SZ; 
} 
«| BSD 03/2013 





the virtual address va of the pages to be dumped is 
extracted; 

¢ the dump_ write(..) method is called to dump the va 
virtual address on the di dump media for the comput- 
ed size sz starting at the offset dumplo; 

¢ all offset and size counters are increased. 


The dump wtite(..) method is only a wrapper for the 
dumperinfo.dumper(..) method that is responsible for the 
physical I/O dump of the data. Usually the dumper meth- 
od is associated to the d_dump field of a GEOM struc- 
ture, and as an example the dump method for the ATA 
disk drives is shown in Listing 19 (extracted from dev/ 
ata/ata-disk.c). The method simply prepares a bio struc- 
ture (Block I/O) that will reference the virtual address 
passed as argument (for example, the page with the da- 
ta to dump), will write on the physical storage no more da- 
ta than the length passed as argument using the specified 
offset. Once the bio structure is ready with the above data, 
it is passed to the GEOM layer startegy routine for being 
processed and enqueued for the physical write. 

As a final note, please take into account that a dump 
header is placed at both boundaries of the dump itself, 
that is at the beginning and at the end. This means that 
on the very last sector of the dump media there will be a 
dump header, and this header is used during the restora- 
tion mechanism (see next section) in order to validate and 
get information about the size of the dump itself. 


The Restoring Mechanism 

As already stated, the savecore(8) program is responsible 
for extracting a core dump and placing it in a safe position 
of the file system. The savecore (8) Command is a userland 
program that has to be run with administrator privileges, and 





Box 1. Debugging Symbols 

When compiling a module like the panic one, or every time it 
is worth debugging a piece of code, the compilation should in- 
clude the DEBUG_FLAGS, that tell the compiler to generate the 
symbol table that the debugger can understand. It must al- 
so be took into account that, by default, a module compilation 
provides optimizations, that is the compiler flags -O2 are used. 
While this is fine for production systems, for develooment/de- 
bugging system this is suboptimal, since such compiler opti- 
mization can produce code that is not “mapped” back to sour- 
ce and therefore is harder to debug. For this reason, it is bet- 
ter to compile a development system with optimization turned 
off, that is with explicit flag -O0. In order to have also the ker- 
nel symbols, the kernel itself must be compiled in debug mode, 
that is the kernel configuration file must have the option: 


eae eses www.mtier.org 


makeoptions DEBUG=-g 
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essentially it performs file operations using the well defined 
system calls for accessing data on the swap device, and 
using the C file routines to access the filesystem. The more 
interesting routine in the program is the DoRegularFile(..) 
which is sketched in Listing 20 and that is called with the file 
descriptor of the swap device and the name of the destina- 
tion file (plus other parameters like the size of the dump). As 
readers can see, the function DoRegularFile(..) performs 
a loop for the size of the dump, and at each step reads a wi 
size of data from the dump device, and writes out the read 
data to the fp file pointer. What is not shown in Listing 20 
is that fp is a regular file pointer or a compressed one (for 
example, fopen(3) or Zzopen(3)) and in the case the saved 


core has not to be compressed (for example, compressed 
= 0) all dump blocks that are zero filled are skipped to not 
waste space. Before the DoRegularFile(..) routine can be 
called, the savecore(8) program has to do several checks 
and preparation: as shown in Listing 21 it must move the 
file offset of the dump device to the where the dump is (re- 
member that a dump is placed at the end of the swap de- 
vice), then a check on the dump headers is performed and 
dump information is printed via printheader(..) (see List- 
ing 8 for an example), as well as the panic string if the dump 
comes from a panic (see Listing 9 for an example). Then a 
reasonable check on the disk space is performed and the 
real work Is done via the above DoRegularFile(..) routine. 





Listing 19. The dump routine for an ATA disk 


Sia ele meniie 
adeCump vOrRd. 4 anc vod. Vals rUcie wi OhmoCtuy Empinyo Meal, 


Cie Ontce r,s) Acme m lencr mn) 


BZSrO (SOO, est ZeOu(Seeuer O1e@)))) 
bpe wowed te ke —ecrer 
bp.bio pblkno = offset / DEV BSIZE; 
If lneetiae = lcirerclag 
Dee biodata —iyiriual; 
Igjesloave: ennlel| = ISN) MSIE IE 29 
ad strategy (&bp) ; 


Se UeIN OOM OMOu cia teeny 


Listing 20. The DoRegulaFile extraction routine 


Static int 
Dokequ are Gntsd, Olt et duns Ze, clam ou Eee once 
Char “device, 
const char *filename, FILE *fp) 
ii Gene p etis mein eel peewee 


Ofte dnpeny; 


dmpcnt = 0; 
he = 0; 
while (dumpsize > 0) { 


wl = BUFFERSIZE; 
if (wl > dumpsize) 
wl = dumpsize; 


ia eer rye tote Wales 


if (compress) { 


nw = fwrite(buf, 1, wl, fp); 





Listing 21. Preliminars steps of the savecore command 


firsthd = lasthd - dumpsize - sizeof kdhf; 
ISGel< (Gil, imesicle, SINS SIE) 
GEOR j= bead (ed, Wckdhn, si7eor kdl 
if (error != sizeof kdhf) { 
syslog (LOG ERR, 
“error reading first dump header at 
Ottsen olidaim osc com, 
(long long)firsthd, device); 


fg eatin 


goto closefd; 


printheader (stdout, &kdhf, device, bounds, 
ic heslgiby oye avels ess bare |||) 
Svelog (LOG TALE a Pewoot atber panic: 
oo je KOM epenrescer ImO) 
else 


syclog (LOG ALERT, “repoor |; 


if (verbose) 
printf (“Checking for available free 
space\n”) ; 
i (teheck ‘space (savedin, “dumpsize)) 4 
nerrtt; 


GOuO -eCloserd: 


if (DoRegularFile(fd, dumpsize, buf, device, filename, 
fp) 
<0) 


Goro scloseall; 
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Conclusions 

This article has presented the FreeBSD kernel panic 
mechanism and when the panic(9) call should be used 
to report a very dramatic condition within the kernel or a 
loadable module. The article also shown how to handle 
a crash dump and to inspect it, assuming a quite luck- 
ily condition where the administrator or the developer has 
the source code and can inspect it with the use of the 
kernel debugger. For this purpose a kernel module that 
simulates a real scenario, where a panic is generated due 
to abnormal conditions, has been implemented to both al- 
low the user to experiment an “on-demand” panic situa- 


tion and to show how to use the panic(9) Call. Finally, a 
few details about how the dumping and restoring of the 
dump data are performed by the system has been shown. 
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Box 2. A simple shell script to launch the kernel 
debugger 


The following script is a simple wrapper that accepts the dump 
number and the dump directory (or set them to zero and dump- 
dir - from /etc/rc.conf — by default) and launches the kernel 
debugger (after having done a few other checks about the file 
existence, the user being root and so on). 


#! /bin/sh 
A Kernel Debugger launching script. 


Usage: 


Stare Kgdb=sh cum number Sl dump din lt] 


Siecle pice lone Om Uns ae clan 


# 

# 

# 

# 

# example: 
# 

# 

# If not specified the dump number is assumed 0, the dump directory 
# 


HSnexERACEeCd ELOM/eGecy re. conm 


+ Check I’m root 


ID="id -u”~ 
if [ $ID -ne 0 ] 
then 


echo “Sorry, you need to be root to execute this script!” 
exit 
fi 


# first argument is the dump number or zero by default 
te Sie Stee 0) | 
then 
DUMP _NUMBER=$1 
else 
DUMP _NUMBER=0 
ot 


# second argument is the dump directory or 
# the Girectory deiined im the re-con: file 
eee ee ae ee 
then 

DUMP _DIR=$2 
else 


ere) tescont 





i a OC Mie cine wl 
then 

DUMP DIR=Sdumpdir 
else 


DUMP DIR=/var/crash 
fi 


fi 
echo “Debugging for dump number $DUMP_ NUMBER in $DUMP DIR” 


+ Huda tiles for Core anc iio 
DUMP FILE CORE=${DUMP DIR}/vmcore.$ {DUMP NUMBER} 
DUMP FILE INFO=${DUMP_DIR}/info.${DUMP NUMBER} 


# check that files exist 

alate [ne eerie SDUMP_ FILE CORE =O. out SDUMP_ FILE INFO ] 

then 
echo “Cannot find dump core/info files!” 
echo “I was looking for SDUMP_ FILE CORE and SDUMP_ FILE INFO” 
exit 

fi 


# get the kernel directory and the debug file 

HOSTNAME=* hostname - 

HOSTNAME="@”S { HOSTNAME } 

KERNEL D=" grep “S {HOSTNAME }” ${DUMP FILE INFO} rchiiskeate Earns? acai 

Ppiediiee oe 

KERNEL D=${KERNEL D}/kernel.debug 

slit a me eee SKERNEL_ D ] 

then 
echo “Cannot find kernel debug symbols!” 
echo “Kernel debug info was $KERNEL D” 
KERN ED 

fi 


# ok start the debugger 
kgdb -d SDUMP_ DIR Sl SDUMP_ NUMBER SKERNEL D 


On The Web 

¢ Source code of the kernel module for generating on-demand 
panics: https://github.com/fluca1978/flucal1978-coding-bits/tree/ 
master/teaching-stuff/FreeBSD/panic_module 

« FreeBSD Documentation on Kernel Debugging: http:/)www.free- 
bsd.org/doc/en/books/developers-handbook/kerneldebug.html 
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Primer — Part 3 


In the third part of our series on programming, we will look at code 
structure, program flow and how to embed CSS and Javascript in 
our pages. We will also start using SQL to dynamically generate 


web pages. 


What you will learn... 
- How to to configure a development environment and write HTML, 
CSS, PHP and SQL code 


basic construction of our programming language 
(PHP), the directory and functional structure of our 
CMS and how this all fits together. 

Our CMS will be designed to be as extensible as pos- 
sible, and will follow the design as detailed in Figure 1. 
Pages will be stored in the MySQL database, merged with 
the header, templates, CSS and Javascript and returned 
to the client browser. This will allow us to separate design 
from content efficiently. 


B efore we start coding in earnest, we will look at the 


Part 1. PHP Fundamentals 
Any language — both verbal and programming — compris- 
es of separate elements that fit together in a logical struc- 
ture that communicates meaning to the recipient. For lan- 
guage to be effective, rules are strictly defined, not only to 
preserve efficiency but to prevent misunderstanding. For 
example, a written sentence will comprise of nouns, verbs 
and adjectives — likewise computer code will consist of 
variables, expressions and control structures. The main 
functional difference between a human language such as 
English and a programming language is flexibility and in- 
terpretation — as humans we are adaptable enough to in- 
terpret a missing full stop or misspelled word correctly, 
whereas a computer will fail miserably. 

Here will will look at some of the the basic building 
blocks of PHP. 
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What you should know... 


¢ BSD and general PC administration skills 


The following code examples can be created in the ex- 
amples directory using your favorite editor (in this case | 
am using VI). Login to the webserver using SSH or from 
the console, switch to the DEV account from root and cre- 
ate the files: 


dev# su 
dev# su dev 
dev# cd /home/dev/data/ 


dev# mkdir examples 


Browser request 
Page = 






Reoord 
1 


mw) ce 





ere, 





Figure 1. CMS design 
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dev# cd examples 


dev# vi examplel.php 


The example can then be run from: http://vourserveri- 
paddress/examples/example1.php (see Figure 2). 

You do not need to run the examples to develop a work- 
ing CMS, but they are useful to experiment with. Change 
values and experiment. 


PHP tags 

All PHP code requires an opening <?php tag. The closing 
?> tag is only necessary when combining PHP with Javas- 
cript, HTML etc. 


Comments 
ln PHP comments are denoted with //. A block of com- 
ments can also be with a block using /* ... */. 


Expressions 

To quote the PHP website “Expressions are the most im- 

portant building stones of PHP. In PHP, almost anything 

you write is an expression. The simplest yet most accurate 

way to define an expression is “anything that has a value”. 
For instance, if the server has 3 disk drives this could 

be written as: 


<?php 
edisk drives = 3; 


Constants 

A constant is useful where the the value will not change 
across the execution of the script. Unlike variables, con- 
stants are accessible within function calls and throughout 
the script, whereas variables may be local only to that par- 
ticular function. This aids program readability. 





Lia 
| 
| 


Figure 2. example!.php running via a browser 
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The circumference of a circle is calculated by multiplying 
the diameter by PI (3.14). As Pl is a constant and will not 
change, we can define PI as a constant. See Listing 1. 


Variables 

A variable holds the result of an expression or statement. 
As the name implies, the value of the variable will change 
over the life of the program. The variable name is defined 
on the left hand side of the = sign and the value of the vari- 
able is defined on the right hand side see Listing 2. 


Data types 

Data types, as the name suggests, define what type of da- 
ta we can hold in a variable or constant. The basic types 
are booleans, integers, floats, strings, and arrays. 


Booleans 

A boolean is used where a dual state is useful. A boolean 
has one of two values, TRUE or FALSE. For instance, 
we can define the constant DEBUG and act accordingly 
depending on how DEBUG evaluated by the “if’ control 
structure: see Listing 3. 





Listing 1. example!l.php 


<?php 


/* 
* examplel.php 

“eons ban rs 

* Define PI as the constant 3.14 and output the result. 


* 


a 


define (“PI”, 
echo PI; 


3.14); 


Listing 2. example2.php 
<7 hip 


We 
* example2.php 
* Variables 
* Define circumference as the variable Scircumference 
with the value 
* V2) rend OuLpuL “tire. result 
x 
*/ 
Scircumference = 12.775; 


echo SCcircumterence: 
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Integers 

An integer is a whole number of the set z = {..., -2, 
-1, 0, 1, 2, ..}. As the maximum and minimum value is 
platform independent, we can access this value via the 
constant PHp int max. If PHP encounters a number larger 
than exp int max, the number will be converted to a float. 
See Listing 4. 


Floats 

The maximum size of a floating point number, like Inte- 
gers, is platform dependent. However, all sorts of round- 
ing and logic errors can be introduced into code with floats 
as they behave differently from integers so particular care 
should be used. For instance, 1/3 is equal to 0.33333 with 
3 recurring to infinity, but as we have limited space it is im- 
possible to represent this fully. If accuracy is critical, vari- 
ous libraries are available to improve float functionality. 
See Listing 5. 


Strings 
A PHP string can contain up to 2Gb of characters. A string is 
limited to 256 ASCII characters, and does not internally store 
strings in Unicode format unlike some other languages. 

A string can be surrounded either by single or dou- 
ble quotes. If surrounded by double quotes, PHP will 


endeavor to evaluate any variables contained within. A 
single quoted string is called a string literal as the string 
is interpreted literally rather than expanding any vari- 
ables contained within it. Like the Vi versus Emacs dis- 
cussion, the use of single or double quotes is very much 
a question of what you want to achieve. While single 
quotes may be considered quicker than double quotes, 
other coding factors have a greater impact on speed. 
See Listing 6. 


Arrays 

An array is a list with a key and value pair. For instance, 
we can list the major BSD distributions as an array vari- 
able, rather than multiple separate variables. We can then 
perform operations on the list by looping through the key 
/value pairs. 

Arrays are useful where we have to keep records in 
sync. For instance if the records from a database table 
were dumped into an array it would be easy to manage 
using the record ID as key. If separate variables were 
used, it would be difficult to manage and the potential for 
errors would be great. Arrays do not need to have sequen- 
tial keys, indeed PHP supports the mixing of numeric and 
string values as keys within arrays. You can even define 
another array as the value of an array 





Listing 3. example3.php 


<?php 


eames ar. 


* Booleans 


+ ie a TEN lias Ss = eee iT ae eel ae an de Ss ha (on) ae 
x Detine DEBUG as a boolean, and print our 


Olosiale(S SIN AWihs wOs Ieiys ke Gy elerciale(S Melts Lie ole whetsrerctelee 


define (“DEBUG”, TRUE); 
if (DEBUG) { 

echo ‘We are in Debug mode’; 
foeiee rt 


echo ‘We are not in Debug mode’; 


Listing 4. example4.php 


<2 io 
ex 
x example4.php 





Ss 2 


Integers 


Check te teeiie Initia tie SC etre ee mel ell eel Otley O.llts 
Akai sO ene 


For a 32 bit system this will be 2147483647. 


echo PHP INT MAX; 


Listing 5. example5.php 
<7 


A@ eal elk Selena) g Sees ce CiligciES 
ee = D9 i > 
LOCMILEe 22. f. 7s 


7 ies VShOukd ase blene Slade ole 6S yl 


Sle eee 
echo Spi; 
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The first array example is the traditional PHP method for 
defining arrays, the second version onwards is available 
in > PHP 5.4. See Listing 7. 


Operators 

Operators are used to compare values (Comparison / Logi- 
cal) or change values (Arithmetic / Assignment / Increment 
/ Decrement / String). Care has to be taken with operator 
precedence, as certain operators will fire first. For example, 
the multiplication operator * takes precedence over the ad- 
dition operator + unless the addition portion of the equation 
is wrapped in brackets: see Listing 8. See Table 1 for the 
full list of of the most common operators. 





Functions 

Functions are small blocks of code that can act as effec- 
tively as a “black box” to the code that is calling it. As func- 
tions can be called repeatedly from anywhere within code 
— and if written properly — will provide a consistent result. 
Functions can act independently of the code that is calling 
them, or can return a result that can be manipulated by the 
main body of the program. An important point to realize is 
that variables defined inside a function are generally out of 
scope of the main body — that is to say $a in the main body 
of a program cannot be accessed by the function unless 
it is either passed as a parameter or accessed via some 
other method (PHP has a rich library of internal functions, if 





Listing 6. example6.php 


<?php 


/* 
* example6o.php 
Sees 
+ Vemonotret lon OL Unie PHP strimdukyoe: 
* Note that the last line is functionally identical to 
the previous 


* line and we are separating each line with a HTML <br />. 


* 


eh 
define ("BR”, ‘<br />'); 
Spi = 22 / 7; 


6cho “Whe value of PI as: Spi’ 2 BR; 
eclies “Whe~ value Om Pils: 99 Sp bee ek. 


echo The value or Pi ts: Spi” = BR; 


Listing 7. example7.php 
<?php 


Te 
* example?/.php 

* Arrays 

* All of these examples are functionally equivalent 


* 


a, 


dering BR *<r =) >” ji: 
// Define the array then print it out using the function 
Picinir ar) 


// Use BR to separate each line 





parray 1 = array ( 
“0” => “FreeBSD”, 
“1” => “OpenBSD”, 
= Nereen 
); 


PEimte (Serra ik, 


echo BR; 


parray 2 = | 
“0” => “FreeBSD”, 
“1” => “OpenBSD”, 
“2” => “NetBSD”, 
|; 


iit (oedcica yah, 


echo BR; 


// Arrays can use mixed key values - they do not have to start at 0 


veigtey eos (P= Ecce oD 
rottay o| Enis is keysG ))— “OpenkoD”, 
reattayes |) je=— NeuBoD ; 


De ie (ates )p, 


echo BR; 


// Let PHP assign the key values 


patray 4[|| = “FrecBsD’; 
earray 4[] = “OpenBSD”; 
sarray 4[] = “NetBSD”; 


jonediate 1a l(Sichiiechy 4) 


echo BR; 
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you do not recognize a function call in the later CMS sam- 
ple code the script will be using a built in PHP function. The 
same apples to the javascript sample). See Listing 9. 


Control structures 

Control structures, along with Comparison / Logical op- 
erators provide the logic for our program. Example 3 is a 
good example of the if/else control structure. 

These are only a very small subset and the most com- 
mon of the extensive features available with PHP. To see 
the full list, please visit the PHP language guide at hitp:/ 
www. php.net/manual/en/angref.php. 


Table 1. PHP Operators 


=> Clan} el(s Name RCSIULLE 


4+ 
mw 


a+$b 
a-Sb 
a*Sb 
a/ $b 
a % Sb 


ir | in 
w wy 
+ II 
II Ww 
oa] 


a= ‘Hello’ 


a ‘world’ 


iH 


9a == $b 
Sa =S==— Sb 
Sa!=$b 


a<>Sb 
Sa !== $b 
a< $b 
a>S$b 
a<=S$b 
a<=S$b 
a and $b 
aor $b 


a xor Sb 


: 


a && Sb 


d 
r 
r 
t 
d 
a|| $b r 


4+ 


+ + 4S} — + 4S 4+ + 4S + + 
+ 4S 
Sr w 
mw 


a++ 


4+ 
. 
I 


Post-decrement 
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Sum of ands 


aoa cfs arithmetic 


TRUE if $a is equal to $b, and they are of the same type 


Multiplication Product of $a and Sb 
TRUE if $a is not equal to $b after type juggling 


TRUE if $a is not equal to $b after type juggling. 


: 


TRUE if Sa is not TRUE 


Part 2. CMS Structure 

See Table 2 — CMS directory structure. Create the di- 
rectories and the 14 files as per the instructions for the 
example code under Part 1 — PHP fundamentals. Before 
we can start coding in earnest, we need to populate our 
MySQL database. 

Create the following files, and create the database, ta- 
ble and our first page stored in the database: see Listings 
10-12. Note that the |lpsum Lorem test should be on one 
line with no carriage returns or line feeds. Your editor may 
wrap this very long line. Create the the database, table 
and page as follows in Listing 13. 


Operator 


TRUE if $a is equal to $b after type juggling Comparison 


TRUE if both $a and $b are TRUE Logical 
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Listing 8. example8.php 


return Spi * Sdiameter . BR; 


<?php } 


x UMC EL OM upiite me IeO7 (Sola Mowe t ae.) 1)) 


* examples.php 


* Demonstration of operator precedence // primencives (); will vdisplay.>pi - diameter 
i // As BR is a global constant and Spi has been passed to our 
a // function we can access them directly. 


// Return our result to the main body of the program 


Celine (2 BR “on > 


return Spi * Sdiameter . BR; 


1+ 5 * 3; } 
Listing 10. createdb.sq/ 


Sa 
Sho] (4+ 5)e > 3; 
create database freebsdcms; 
echo “sca willvevaluate to Tow “ < Sa 2°BR; grant usage on *.* to bsduser@localhost identified by 
echo ‘Sb will evaluate to 18: * . Sb . BR; ‘cmsdbpassword’ ; 
grant all privileges on freebsdcms.* to bsduser@localhost; 
Listing 9. example9.php 
Listing 11. createpagetbl.sql 


<2 plo 
CREATE TABLE if not exists pages ( 
es id INT NOT NULL AUTO INCREMENT, 
* example9.php PRIMARY KEY(id), 
S —DENOnseraeroOn One  HUNCeGE BOG Il title VARCHAR(50) NOT NULL, 
. hl VARCHAR(50), 
vs body TEXT 


We 


// As BR is a constant, this is available to our function directly 


Listing 12. createpage.sql 


define (“BR”, ‘<br />'); USE freebsdcms; 
INSERT INTO pages 
// Spi is not available to our function, we will need to VALUES ( 
access it by ee 
// other methods ‘My first page’, 


‘Page header’, 
Sl eye ‘Lorem ipsum dolor sit amet, consectetur adipiscing elit. Mauris 


interdum auctor tellus sed dignissim. Phasellus non orci massa, 


Selle “Cae biinicsinsaiee: Wakely ch Clr elise S20 joven es oie 5) 7 nec feugiat sem. Vestibulum molestie interdum 

lela, Gseewinesiesiars Welle. Clbeisicsie IY. > joreibiane verlcer, (10) yer) bibendum. Nunc guis elit nulla, sit amet rutrum lorem. 
Quisque 

EDUONCNE ONel jovettiae (Caio Siclleniie cieia ) | odio est, Sagittis nec accumsan ut, placerat sit 


amet lectus. Curabitur aliquam dignissim felis, a malesuada leo 
// print circl() will display $pi * $diameter fringilla at. Sed ornare aliquet lacus, quis 
// Define Spi as a global variable imperdiet augue mattis eu. Nulla porta odio ut erat 

CONSE Ceenmte wane 

gi¢bal “Spi; molestie justo suscipit. Aenean convallis 

pellentesque nisl, vitae posuere mauris facilisis vitae. 
// As BR is a global constant we can access it directly Morbi in 
// Return our result to the main body of the program tellus nisl, vel facilisis diam.’ 


he 
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Part 3. The Code 
The following PHP files contain the code for out website. 
Create each file as per Table 2. See Listing 14. 
global.css holds the style information for our site. Ex- 
periment with the font sizes, line spacing etc. to style the 
site to your liking. If you use Firefox, install the Firebug 
plugin to dynamically change the values, but if you want to 
make them pernanet you will need to edit this file. Also try 


F 5 ptr ape 
-+ ¢ ay, Meh Lk a 


4) PAGE HEADER 


Lone rh Oe Pe CO OO CR Mae Ie ee OR A Ce. PE ce) OC) ee, ee a WE ee ET) 
Berea. Mare Gal £02 fk, GE beret PUP afer. (Ru Ss i, ee A , ee ie, Ce ee Cre Ie, A Pi es 
Priep o2. ad Greed Aue! lieSurl, Guee rp degen Pa de. Pl pee cede ul eet cereale oe ee fe deep. Re Corel Geer fel, wld Godoere 
Buu Cee ie. Met im tees ee, ed Le ce 


Figure 3. Our first page — index.php 


renaming global.css and refreshing your browservs cache 
to see the effect of the styling on the site. See Listing 19. 

The include files build our basic HTML header and foot- 
ers, add the CSS add the CSS via global.css and load 
the Javascript. See Listing 20-22. The javascript files 








Figure 4, The page validates 





Listing 13. Creating the database, table and pages in MySQL 


#dev mysql -u root password ‘cms-password’ < createdb.sql 


#dev mysql -u root password ‘cms-password’ < createpage.sql 





#dev mysql -u root password ‘cms-password’ < createpagetbl.sql 








Table 2. CMS directory structure 


CMS Directory structure 
All directories are under /usr/local/www/apache22/data 


Directory / file Purpose 


examples Example PHP code 


includes 


index.php 


Start page for our CMS 


Javascript support for our website 


Contains the SQL loader scripts for our website. 


javascript 


sql 


stylesheets 
templates 
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PHP includes for CMS. Each file contains specific functionality as named 


examplel.php 
example2.php 
example3.php 
example4.php 
example5.php 
example6.php 
example7.php 
example8.php 
example9.php 


cms.inc 
core.inc 
html.inc 
mysql.inc 


index.php 


postload.js 
preload.js 
createdb.sql 


createpage.sq| 
createpagetbl.sql 


Holds the CSS stylesheets for the website global.css 


Holds the templates for the website 


header.inc 
footer.inc 
template.inc 
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Listing 14. index.php 
<?php 
/* 

ACS x sOmn> 


* Index page for FreeBSD CMS 


* 
a 
// Get required files 
// Our global settings - Note need full path 
require once ‘includes/cms.inc’; 
// Core functions 


Bequ ras Once wtih DES Cone me | 


(Poti fume t ets 
Gequrre Once INCLUDES. ume ame 7 


// Mycol, Funcerons 
require once —INChUDESs “mysqiane 7 


// Turn full debug functionality on if enabled 


if (DEBUG) { 


// Turn on full PHP error reporting 


SLrOr eeporring (i Alin); 


telsef{ 


// Hide all error messages 


SEGoOr er porting (Or, 


// Build page - use first record in database 
Spage[‘id’] = 1; 

buildpage (Spage) ; 

Listing 15. cms.inc 

<?php 

/* 


* 


ems Sane 


* Contains Cerauleyselteimgs for our CMs 


* NOTE: {@ denotes a line wrapped - all code should be 


on one line 


* 


sh 
// Set our timezone 
date default timezone set (‘Europe/London’ ); 
// Copyright details 
define (“LICENCE”, ‘licence.txt’); 
define (“COPYRIGHT”, ‘Copyright &copy; 2013 Rob Somerville 
q me@merville.co.uk’); 
define (“COPYYEAR”, date(‘Y’)); 
define (“COPYAUTH”, ‘Rob Somerville’); 
define (“COPYEMAIL”, ‘me@merville.co.uk’); 
// Version 


define (“VERSION”, ‘Version 1.0 not for production use’); 


// Mode - If DEBUG is set to true, show errors and debug 


HET) 
define (“DEBUG”, TRUE); 
// Where to find our files 
define (“TEMPLATES”, ‘templates/’); 
define (“INCLUDES”, ‘includes/’); 


define (“SQL”, ‘sql/’); 


// HTML tags that are orphaned and not defined in out 
template files 


detine( “BODN Fy “<body>” ); 
detine (“HEAD”, “</head>’ ); 


// MySQL details 


define (“DBSERVER”, ‘localhost’ ); 

define (“DBUSER”, ‘bsduser’); 

define (“DBPASSWORD”, ‘cmsdbpassword’ ); 
define (“CMSDB”, ‘freebsdcms’ ); 
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Listing 16. core.inc 
<7 pho 
/* 


* core.inc 


TUCO thcueOre. KUNCEToOMs, Pom "One TMs 
* 
zi), 
function buildpage(Spage) { 
// Builds a standard page 


Sid = Spage[‘id’]; 


// Build the SQL and get the result 


Ssql = “SELECT * FROM pages WHERE id='Sid’ LIMIT 1”; 


Sresult = mysql select ($sql); 


// Outout oum page neader 


outfile (TEMPLATES ‘header.inc’); 


/7/ Creace our body 


Smarkup = ‘'; 

Smarkup .= wraptag(‘title’, Sresult[4]); 
Smarkup .= HEAD; 

Smarkup .= BODY; 


// If we are in debug mode, show an alert 


if (DEBUG) { 

Sdebug = ‘é&para; '; 
jelse{ 

Sdebug = ‘'; 


// Add to markup 


Smarkup .= wraptag(‘hl’,Sdebug . Sresult[3]); 


smarkup .= wraptag({‘p’, sresult([5]); 
Smarkup .= divclass(ahref (COPYRIGHT, LICENCE, 
NCO pyc gine ances 


licence details’),’licence’); 


// Output all our markup 


echo Smarkup; 


// Output our HTML page footer 


outfile (TEMPLATES MPOOEeE ALnie. i 


function outfile (Sfile) { 


// Outputs template file to browser e.g header, 
footer, license etc. 


Sfh = fopen(Sfile, ‘r’); 
while (!feof(Sfh)) { 
echo fgets (S$fh) ; 


he lose (sith) 


Listing 17a. html.inc 
<?php 


es 
* 

= Teme ene 

= Contains ore Him: tunciwions= For our MCMs 


* 


t 


function wraptag(Stag, Stext) { 


// Wraps Stext with compliant tags 
// wraptag(‘p’,sometext) 

// <p>sometext</p> 

Stext Ee Sitaq & 


rorurn “<" Stag = 


huNneE Lon divelacse (Sdivcontent, sclass, otc — — 4) | 


// Generates a div tag Stext with compliant tags 


i rcinvcilacs(UcOnrent class” ) 


Ney 
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Listing 17b. html.inc 


ie divaclasc= Clase = Comtont<) ary 


jy Pnelessii Gonrent’. solace, aici } 


diy ela ss— Mkeonce” kO= ale Common) Cle 


ioc ae ue 


Sie = d=! Sic ee 
} 
Perri <Giy ollaga—o 4 Selass Vi Sid 
Sdivcontent oy cay a 


fFUNeCELON anrer (Stext, suri, Stitile =)” )) 4 


// Generates an href tag Stext with compliant tags 


jy) vanrem( Clack Here’, treebed.org) 


i / -<ea Neet=(niiot//tirecbsdrorg? vitle="Click 


hewe >Click here a> 


// ahref(‘Click here’,freebsd.org,’Link title’) 


// <a href="http://freebsd.org” title="Link 


eae © ibe eeinence) a> 


if ($title == ‘’) { 


Stitle. = Stext- 
} 
Sanrer = “<a hrer— ” Sian i rot 
MAAS Std , tT 


Stext a 


Rotini s camer. 


Listing 18. mysq]l.inc 
<7 


re 
x 

2 eC een ean 

ICOniGeinis ly oOlle Lumen KOMS Mero sou CMS 


* 


ou 


HUMe ELOmMe thy sc so hoc iat cet 


Siete lle 


Sdob = new mysqli(DBSERVER, DBUSER, DBPASSWORD, CMSDB) ; 





ii(odbe commeer errmnc. |) 
die(‘Unable to connect to database [‘ 


Selo Cinna See Staicoie 


it (lSresut = 
if (DEBUG) { 


Sdb->query ($sql) ) { 


die(‘There was an error running the 


Sdb->error . 


// Pass our results to an array to be retur 


Sr = array(); 
oe ie Gesu le— nome cone, 
Sx[] = $db->affected rows; 


e.g. J 
update /delete 


White (2 fOw =) oresulL—- PoeCci assoc |) 4 


Set) = Siceuw | wel! | 
aa Owe ile, 
Sip = Low) title” |e 
or t=" > trowl| bedy 


// Eree tine. result 


Srestle--treei)-; 


return Sir? 


Sie 


query [° 


ned 


Lj Nonor Lows —returneo 


/ 2 NOVO ab OWwsecdr hee@red 
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Listing 19. global.css 


/* global.css - the site global stylesheet */ 


ile 
background-color: teal; 
float: left; 
color: white; 
Baccano 2 lox, 


text-transform: uppercase; 


float: Nett; 
} 
body { 
line-height: 160%; 
text oaliom: . jst ict y 
float: lett; 


her | 
background: none repeat scroll 0 O #F9F8F2; 
border: px solid; 
color. teal? 
fome—remmelyss Verdana: 
Maou = Om: 
Paceding.. «Z0jos<, 
} 
.jstime, .licence { 
background: none repeat scroll 0 O #EDEAC6; 
border: lpx solid #DADADA; 
color: slategrey; 
iOa s arenoiEs 
font-family: Verdana; 
fOnE-Sizez x-Ssmall; 
Mat Cin-SOr EC: Spx; 
MacgiIn-rignt:  h0ipx: 
Margin=uop:, 0px, 


paddange op l0ax: 


Listing 20. header.inc 


J CCIy PEaitminPURi es 7 Wee, Dib iis piece: 4) 
ENG Nip.) / Wiis oso IR) chim bib) ecm tS state. aid 
<html xmlns="http://www.w3.org/1999/xhtml” xml:lang="en”> 
<head> 
<meta http-equiv="Content-type” content="text/html; 
Charser— aso- ge59-l" 7 7) > 
<link rel="stylesheet” type="text/css” 
href="/stylesheets/global.css” /> 


<script Sre—"/javascript/preload. ss 4) 


type="text/javascript”></script> 


Listing 21. footer.inc 


<script src="/javascript/postload.js” type="text/ 


Favecckipt ——/ common <body < heml> 


Listing 22. template.inc 


This can be empty, but needs to be created. 


<!-- Template file --> 


Listing 23. preload.js 
/* 
preload.js 


Provides Javascript support 


c 
// Call the function displaydate() 


displaydate () 


function displaydate (){ 
// Displays the date and time in AM/PM format. 


var currentTime = new Date() 

var hours = currentTime.getHours () 

var minutes = currentTime.getMinutes () 
var month = currentTime.getMonth() + 1 
var day = currentTime.getDate () 


var year = currentTime.getFullYear () 


if (minutes < 10) { 


minutes = “0” + minutes 


NW o 
’ 


var ampm = 


ii CMOUneS se-= lies 


ampm = “PM” 
a ean 
ampm = “AM” 


document swrite( <div cllass—-\"jeuime\ >" + day + /" + 


month + “/” + 


Veo e ee er OUIGe et Se Mee ome TOM ym cline) 
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Listing 24. postload.js 
/* 
postload 7s 
Just an empty file with comments 


ay 














Useful links 
W3C Validator (by file upload) — http://validator.w3.org 
PHP documentation — http://www.php.net/manual/en 
W3 Schools — http:/www.w3schools.com 











are split into 2. Preload.js provides the date and time on 
each page, postload.js is just an empty file which provides 
hooks we will use later on in the series. See Listing 23-24. 


Part 4. Our Simple CMS 

Once you have entered the code as per table 2, point 
your browser at http://vourserveripaddress/index.php. 
You should see a page similar to Figure 3. Turn debug off 
and on in cms.inc, and the paragraph mark should disap- 
pear. If you copy the HTML source from the page (In you 
browser view source, select all, copy and paste into the 
W3C validator) the page should validate. 





NAPOLI 





CERTIFICATION.ORG 


w/ BSDA Exam 


So what is our code doing? 

The unformatted text is stored as plain text in our data- 
base table. Index.php forms the first page of our website, 
and loads our settings and functions from the include files. 
The first stage is to load our header from a plain text file, 
which is the HTML at the start of our page. The header file 
in turn loads the CSS and javascript, and returns control 
to index.php. We then query the database, wrap the text 
in the relevant HTML tags and output to our browser. We 
then close the HTML with our footer HTML. In the next 
part of our series we will develop the CMS further, pass- 
ing parameters via our browser to load different pages, 
We will also start using our template file so that we can 
design our site the way we want it with separate blocks 
and regions. 
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SECURITY 


Hardening FreeBSD 


with TrustedBSD and Mandatory Access Controls (MAC) 


Part 5 


Most system administrators understand the need to lock down 
permissions for files and applications. In addition to these 
configuration options on FreeBSD, there are features provided by 
TrustedBSD that add additional layers of specific security controls 
to fine tune the operating system for multilevel security. 


What you will learn... 
¢ Configuration of the mac_ifoff, mac_portacl, and MAC LOMAC 
modules. 


tensions have been included with the default in- 

Stall of the operating system. By default, this func- 
tionality is disabled and requires support to be compiled 
in or kernel modules to be loaded at boot time. For the 
purpose of this article, support will be loaded in with ker- 
nel modules already available with FreeBSD 9.1. Part 
5 of the TrustedBSD series will cover the basic config- 
uration of the mac ifoff, mac _portacl, and MAC LOMAC 
modules. 


S ince version 5.0 of FreeBSD, the TrustedBSD ex- 


Warning 

Incorrect MAC settings can cause even the root user to not 
be able to login to the system. Be sure to run these tests on 
a VM or test machine to avoid any issues with production 
systems. This article assumes that a fresh install of Free- 
BSD 9.1 has been performed before continuing. 

As in the previous articles, a certain set of users will help 
to illustrate how to use mandatory access controls (MAC). 
For the mac _ifof£ module, the focus will be on network in- 
terfaces for the operating system. Listing 1 shows a basic 
setup for the required users for this article. 

The mac_ifoff module provides an interesting feature 
that can turn off the network interfaces based on a sysctl 
value. To enable the mac_ifofft module add the following 
tO /boot/loader.conf as detailed in Listing 2 to load the 
module at startup. 
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What you should know... 
- Basic FreeBSD knowledge to navigate the command line 
¢ Familiarity with loader.conf to enable kernel modules at boot 


Once the system reboots, you will see “Security polli- 
cy loaded: TrustedBSD MAC/ifoff (mac_ifoff)” in the out- 
put of dmesg. Manipulating the sysctl values for “mac_if- 
off’ allows for the network interfaces to be enabled or 
disabled. In order to make it easier to demonstrate this 
module, tmux will be installed to move between different 
user screens. The basic usage of tmux will be covered 
with additional information provided in the reference sec- 





Figure 1. Using tmux and multiple window panes 
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tion. Listing 3 shows the installation steps for tmux using 
FreeBSD ports. Once tmux is installed, type tmux to in- 
voke the terminal multiplexer and type the following: 


Cirl-p: * 
Corlab, ” 


This will open two window panes in one terminal, as 
seen in Figure 1. 

Using Ctrl-b and the up and down arrow keys allows for 
the movement between window panes. Move to the top 
window, and run ia then move to the middle pane and run 
ssh userl@localhost. Once logged IN aS userl type Ctrl- 
b and the arrow keys to select the root terminal and type 
what is shown in Listing 4. 

When writing this article, ssh was used to remotely con- 
nect into the virtual machine with tmux. Figure 2 shows the 
result of both the root and user1 ssh connections being 
terminated immediately. 





# sysctl security.mac.ifoff 
security.mac.ifott.bptrecv_enabled: 6G 
security.mac.ifoff.other enabled: @ 
security.mac.ifoff.lo_enabled: 6 
security.mac.ifoff.enabled: 0 

# sysctl security.aac.ifoff.enabled=1 












If you still have a question or problem. please take the output 
‘uname -a', along with any relevant error messages, and email it 
as a question to the questions@FreeBSD.org mailing list. If you 
unfamiliar with FreeBSD's directory layout, please refer to the 
manual page. If you are not familiar with manual pages, type ‘a 


Edit /etc/motd to change this login announcement. 





You can disable tcsh's terminal beep if you “set nobeep’. 
TET ee alee ee 


Figure 2. mac_ifoff being enabled killed all network connections 







RL i ee ee ie Fe A) 
PING 192.168 .57.1 €192.168.57.1): 56 data bytes 
ping: sendto: Operation not permitted 











132 .168.57.1 ping statistics 
1 packets transmitted, © packets received, 100.0% packet loss 
Bee ee ee ee ee ete 

Ssecurity.mac.ifoff .enabled: 1 ->) © 

@ ping -c 1 192.168.57.1 

we ee Fe es eee ee ee es ee eb 

b4 bytes from 192.168.57.1: icmp_seg=0 ttl=64 time=0.386 ms 















--= 192.168.57.1 ping statistics --- 

1 packets transmitted, 1 packets received, ©.0% packet loss 

round-trip minvavg/max/stddey = 6.38670. 38670.38670.000 ms 
L 






Figure 3. mac_ifoff being disabled which allows network traffic 


www.bsdmag.org 


Become BSD magazine 


TMC Imo laldal oleic. 

to BSD magazine, share 

your knowledge and skills with 
other BSD users — do not 
hesitate — read the guidelines 
on our website and email us 
your idea for an article. 


Join our 
team! 









Author or Betatester 


PNM LACH -d-1 Cm e) bore] 
decide on the contents and 
the form of our quarterly. 

It can be you who read 

the articles before 
everybody else and sugpest 
the changes to the author. 


TTC foa Ee 
editors@bsdmag.org 
www.bsdmag.org 


SECURITY 








Listing 1. Users setup on FreeBSD 


# pw user add -n userl -s /bin/csh -m 
# pw user add -n user2 -s /bin/csh -m 
# pw group add user-reg -M userl,user2 
# passwd userl 

Changing local password for userl 

New Password: 

Retype New Password: 

# passwd user2 

Changing Neocal password for vserZ 

New Password: 

Retype New Password: 

# groups userl 

userl user-reg 

# groups user2 


user2 user-reg 


i 
Listing 2. Loading the mac_ifoff module on system startup (mac_ 
portacl is added here as well)... 


7 echo. ‘Mac iteftt Voad="VYES"’ >> /boot, loaders cont 


# echo ‘mac portacl load="YES”’ >> /boot/loader.conf 


# echo ‘security.mac.ifoff.lo enabled=0’ >> /etc/sysctl. 


COneE 

# echo ‘security.mac.ifoff.enabled=0'’ >> /etc/sysctl. 
COne 

# echo ‘security.mac.ifoff.other enabled=0’ >> /etc/ 
Seu eo rus 


# reboot 


Listing 3. /nstalling tmux 


+ Cd /USt/pores/sysutals/ tmux/ 

# make -DBATCH install clean 

===> License BSD accepted by the user 

===> Found saved configuration for tmux-1.7 1 
Olle Ouiewe 

===> Cleaning for tmux-1.7 1 


if 


Listing 4. Enabled the mac.ifoff module 


# sysctl security.mac.ifoff.enabled=1 


Listing 5. Turning on the mac_portacl module which allows user] 
to bind to TCP port 80 


(In the root window, type the following) 

7 (SyeCe lM ccCUrIEY. Mac. pOtracl. pore mrgi—l073 
SSCUMULY Mac spe abacl poutine: 07> a —>h023 
# sysctl net.inet.ip.portrange.reservedhigh=0 


net.inet.ip.portrange.reservedhigh: 1023 -> 0 


# id userl 

uid=1001(userl) gid=1001(userl) groups=1001 (userl1) 
# sysctl security.mac.portacl.rules=uid:1001:tcp:80 
Seeley Mae. DOrkdel. rilles: => id: N00 ceo s0 


# 


(Use Ctrl-b and the arrow keys to move to the userl 
window and type the following) 

= ey = nee oO 

(GOO eecane connect suo thts pore locally wath tic wm 


0 0S 0) 


Listing 6. Enable labels for the root file system (Warning: these 
commands are based on a default install with a single root partition 
and swap. If there are multiple partitions, edit the /etc/fstab by hand 
to ensure the root partition is set to ro) 


+ sed. 9’). \s/rw/ co)” ere) fstap 

# reboot 

(Type 6 when it reboots to go into Single User Mode) 

# tunefs -1 enable / 

# reboot 

(Let the OS boot normally) 

# mount -urw / 

# sed -i ‘’ -e ‘s/ro/rw/' /etc/fstab 

(If there are multiple partitions, set root back to 
readwrite ‘rw’) 


7 Ge OOOL 


Listing 7. Setup of files for mac_lomac 


(create some files) 

# mkdir -p /data 

# echo “HIGH” > /data/high.txt 
# echo “LOW” > /data/low.txt 


(change files low.txt and high.txt to lomac/high[low]) 
# setfmac -R lomac/equal /data 

# setfmac lomac/high /data/high.txt 

# setfmac lomac/low /data/low.txt 

# setfmac lomac/high /usr/bin/vi 

+ Vie) dake, Low. exe 

ex/vi: Error: Log file: Permission denied 


it 
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Hardening FreeBSD with TrustedBSD and Mandatory Access Controls (MAC) Part 5 


The functionality as highlighted in the FreeBSD hand- 
book could be combined with an integrity checker such as 
aide or some other script. For example, monitoring /etc/ 
shadow for changes to passwords and disabling remote 
connections when discovering an unauthorized change is 
a variation of checking the integrity of the system. From 
the console on the VM, Figure 3 shows how the interfaces 
are disabled until the sysctl value is changed to 0. 

The mac_portacl module has interesting use cases for 
when administrators wish to allow under privileged us- 
ers access to bind to privileged ports. This is beneficial 
for applications that do not provide a mechanism to drop 
root privileged. The mac portacl module was added to 
loader.conf in an earlier procedure and should be load- 
ed. Refer to the previous steps for launching tmux and 
using Ctrl- b” to create a window for root and user1. Run 


% nc -vnl 80 


nc: Operation not permitted 


2 
LJ 


Tial a:esh* 
Figure 4. user] trying to bind to port 80 


Cart eee ea ee ee ee ee ee ee 

eae ee ae Ceol ee Pa ee tc ta 
az sysctl net.inet.ip.portrange.reservedhigh=0 
Enet.inet.ip.portrange.reservedhigh: 1023 -> 0 

# id userl 

suld=1l00l(userl) gid=100l(userl) groups=1001(userl) 

sysctl security.mac.portacl.rules=uid:1001:tcp:80 

isecurity.gac.portacl.rules: uid:1001:tcp:87 -> uid:1001: tcp: 80 
1S ee Te ee ae = 
iS | 


c -ynl 80 
: Operation not permitted 
t nc -vnl 80 


ae 


Figure 5. Output from allowing user! to bind to port 80 
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nc -vnl 80 as user1 and verify the output is the same as 
shown in Figure 4. 

nc stands for Netcat, one of the most versatile network 
tools. Using the features of the mac portac1 module, user1 
will be able to bind to the privileged port. Listing 5 contains 
the sysctl values that enable the module. 

Figure 5 shows the output after allowing user1 to bind 
to port 80. 

The last of the mandatory access controls builds upon 
the previously discussed mac biba and Is Called mac _lomac 
module. This module extends the mac biba module. Us- 
ing previous steps from a previous article, we setup the 
proper labeling on the root filesystem and loading of the 
mac _lomac module at boot-up. Listing 6 contains the nec- 
essary steps for this process. 

Listing 7 contains some commands to test the mac_lo- 
mac settings using Vi. 

In this example, vi is set to a lomac/high profile, which 
does not allow it to continue opening the file once it ac- 
CESSES a lomac/ low file. All of the information presented in 
these articles is a start to providing examples on how to 
implement Mandatory Access Controls with TrustedBSD 
on FreeBSD. Even as the FreeBSD code base moves into 
version 10, these features are still labeled as experimen- 
tal and are to be thoroughly tested before deploying in 
production. Future security enhancements may incorpo- 
rate these modules or replace them with newer code to 
better fine tune access controls on FreeBSD. 
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In the next issue: 
ALL ABOUT FREEN AS! 


¢ Article by John Hixson, author of the 
FreeNAS plugins 

¢ Dru Lavigne article on plugins and 
encryption in FreeNAS &.x 

e Interview with Alfred Perlstein, 
Vice President, Software Engineering 

¢ Article by Mark VonFange, Professional 
Services Manager at iXsystems 
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Paws Studio Is efficient, easy to use and cost effective. The 
software provides comprehensive reporting and management 
summaries to appeal to all levels of your organization. 
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Antivirus 
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Software Updates 
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